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ABSTRACT ^ 

Presented at a symposium -instigated to improve the 
accessibility and usefulness of educational technology, the five 
invited papers presented in this' proceedings deal with various 

' aspects of information systems and their relationship to each other. 
Topics covered were "Educational Technologists as. Consumers of 
Educational Information", "Information Services in Practice", "The 
Use of Query in Multi-fafcet Indexing of Information Materfals*, "The 
'Exeter Abstract Reference System in Higher Education", and "Qualified 
Citation Indexing-^-I ts. Relevance to Educational Technology." Each 
paper is accompanied by an invited structured response • Also included 
in the report are responses of participants invited to complete a 
practical Exercise on identifying individual needs in information 

* retrieval, ahd summaries of the discissions pertinent to, that 
exercise, and to the theme of the symposium, (MER) . * 
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Summary 



The Symposium on Information Retrieval in Educational Technology 
held at ETTC ! 8l in April 198l was instigated by a desire to improve . 
the accessibility and usefulness of educational technology resources. 

The format of the Symposium, was the presentation of five invited 
papers on various aspects of information ^systems and their relation- 
ship with users. These papers, together with invited, structured 
replies, are printed in full in this Monograph. 

la addition to the papers, a Task was presented to all participants 
to think about and complete, with the dual purpose of directing 
thinking, along information-seeking lines, and of producing a basis 
for further discussion during the day. The nature of the Task, and 
a brief analysis of 'responses to it precede the text of the main papers. 

Eie third section of the Monograph is a summary from discussions, which 
were directed in three groups one bn each of three important aspects 
of th$ day T s theme. * 

Ihe Monograph is introduced by a general chapter on information 
retrieval in educational technology, which* is an overview of poifl^s 
which arose from discussions throughout the day and ways in which 
future systems might be, planned in the light of user reactions tfo 
the status A quo. 
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CHAPTER 1 -» 

INFORMATION RETRIEVAL IN EDUCATIONAL TECHNOLOGY f> 

E B' Duncan, Robert Gordon's Institute of Technology 
R McAleese, University of Aberdeen 



■ This is a summary of events which took place during 
' a One-Day Symposium on Information Retrieval, held on 
1 April, during ETIC'81,. The Symposium Programme is 
listed in the Appendix (Page 88). 

Thirty-three delegates, with varied background interests 
listened to and discussed papers ranging from information- 
seeking behaviour of educational technology users, through 
- - present provision of information services; 

, « to possible innovations in relaying of ^information to users. 

The theme of the day's discussions was the variety of means 
' * by which one can convey information from sgurqe tp. user and 

consideration of ways in which inforrtatiorr might be 
*< oackaged or filtered to achieve this most effectively. • t 

Educational technology as a subject is an area in 
which there has been a great deal of controversy in 
\ identifying who the users really are. It is an area 

' in which resources are scattered, due to its disparate 

nature, and access to them is difficult. It is therefore 
' v important that both the users and their patterns of 

/ information need are clearly identified. 

The format of the'dky was that of a structured Seminar, , 
that is, the presentation of five papers, each followed .by 
a formal, prepared Peply. The objective tff the Reply was 
to stimulate discussion and to H .pose a contrasting view. The 
s presentation of paper's was preceded by an information- 

^ gathering /exercise or task completed by delegates, * 

4 Th§ day* concluded with thfc formation of three Working 

■ Groups discussing separately three topics isolated. as 

being of direct current interest. Each of the three - , 
groups reported back to the Symposiuitl before further 
discussion took place. 
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TASK ^ 

The information need problem was 1 explored in a 

practical exercise or TASK. Participants were asked 

tp complete, in a space of 10 minutes, a set "of questions 

aimed *at identifying individual needs in information T 

retrieval. (See Figure 1) They were asked to identify 

a recent instance where" they had needed to find out. something, 

relating to educational technology T and to indicate from 

a list of ten possible courses of action whic^h would seem 

the most appropriate. Results from this instant, 

assessment were tabulated and used in discussion later in 

the day. (Table 1) 



SUMMARY OF TASK RESPONSES 

Prom the tabulated responses it is clear that most users 

were looking either for someone -who knows the subject, or for a 

background review of the subject. Is information provision 

too abundant? Are filters needed? Compare these categories 

with the fact that: very few people wished to be given a list 

of more than 100 bibliographic items. Is the most popular category 

really the most needed, or is it the most frequently thought of? 

It is difficult to identify # on such a brief response whether 

the replies deferred to 'need'* or 'use', but, a similar, 

informal survey carried out by the 'Qualified Citation Indexing 

Project of approximately 25 potential users from differing 

educational experience (unpublished)' gave>an almost parallel 

pattern of response.' (See" Chapter 6 for details of the paper on 

qualified citation indexing). 1 
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ETIC'Sl 

\ INFORMATION RETRIEVAL IN EDUCATIONAL TECHNOLOGY 

SYMPOSIUM, 1 APRIL 1981 * . * 



TASK: Identifying individual needs in information retrieval 



Please complete the following to the best of your ability 
using an instance from your own experience if possible. 



NAME: * , 

ORGANIZATION: 

, » * 

TASK: 

Think of a recent instance where you have needed to find out 
something related to educational technology. 

Would your need have been met by having one of the following 
solutions suggested to you f 

(Mark with a * any which seem appropriate to your situation) 



[1] 
[2] 
[3] 
[4] 

[-5] 
[6] 
[7] 
[8] 



[9] 
1101 



An introductory work or textbook r 

A background review article ^ 

Someone vho knows something' about the subject . 

Someone who knows something about the literature 

of the subject v - 

An Organization where the -subject «is being studied 
Courses or Conferences which might be relevant 
Software or other *AV materials as well as 'literature 
A selected list of literature and/or software to 

suit your needs y 

A. selected to 2 or 1 items * * 

B. selected to 30 items 

* C. selected to 100. items 
, D. a complete list of all published material 
Names of Professional Bodies in* the subject area 
Any other solution - please specify: 



Mark the most important with an extra * 



q ' Make a brief note of what the topic was: (e.g. distance learning 
J^J^J^Qjj recent research - UK only) 

MMjWffifltiH " Figure 1 : ,TASK -}u,estionnaire 



(1) An introductory work or textbook 

(2) a background review article 

(3) Someone who knows the subject 

(4) Someone who knows the literature 

(5) An Organisation in the field 

(6) Courses or Conferences ' 

-4 

3 (7) Software or AV materials and literature 

(8) Selected list of literature and/or software 

a) selected ^to 2 or 3 items 

b) selected to 30- items \ 

c) selected to 100 items 

d) complete list of all published 

(9) Professional bodies in subject field 
(10) Other solutions 

Committees 2 
Personal files 1 
x Abstracts 2 w 
Workshops 3 / 

Trade literature 5 
, Popular press 1 f -« 

Interview/questionnaire 2 N 
List of research projects 1 



Score 


Rank 


( 8 


8 




2 




*1 


17 




17 


3 m 


.9 


7 * 


10 


6 


4 


10 


14 


5 


2 




4 * 


io 


5 


9 


(19) 
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Table 1: TASK Results 



PRESENTATIONS 

The Dresentations consisted of two general papers on information 
need's and possible solutions. These were followed by reports 
on two working information systems, and finally, a report on 
a new form of' information retrieval in educational technology, 
qualified citation indexing. 

Information needs: \ * *' 



The Present information-seeking behaviour users- 
as displayed by a 'small group* of American University 
lecturers was presented ip the first paper of the day, 
given by- Professor Donald Ely, of Syracuse University 
(Director, ERIC Clearinghouse on Information Resources). 
Professor Ely's paper was on Educational Technologists 
as Consumers/of Educational Information, and described 
the information-coping strategies of educational ^ * 
technologists, the types of educational technologist, 
and the purpose of seeking information. He emphasised' 
in' particular the danger of designing complex systems for 
non-existent people in hypothetical situations with 
unknown needs. The ideal is not to aim, for one 
universal system, but to identify, coordinate and^ 
communicate the information options already in existence, 
> and make them available to the individual user. 

In reply, Mr- N Rushbv of Imperial College, London, emphasised 

the need for identification of real users. There' is a possible 

confusion in people's minds between need and use. 

He asked to. what extent participants cortside?red that 

users' needs are shaped by what is available. 

Points raised in discussion reinforced thg, 

connection between users' nelds and environment, an* 

emphasised the difficulty of communication of need. 

Intermediaries wer considered to be important, a 

point which was also brought out by the TASK results, as 

discussed earlier. The question of what people want is 

often a function of the way in which the question is, 

asked. People like problems - few like solutions. 

Possible solutions to the infor.mat ion-handling problem . • 
* were described* in the paper given by Miss Jane Hustwit^ 
of -the Council for Educational Technology. Miss Hustwit 
outlined some of the work of the Council in investigating 
problems surrounding and blocking access to information. 
In particular she emphasised the role of the "Council in 
the development of experimental information services and 
the investigation of the application of new technologies 
to these problems. The main theme of her paper,- however., 
was a brief description of the CEDAR ptoject on educational 
computing, centred at Imperial College, London, and a 
detailed account of CONTACT, the Council's information 
service on -teaching and learning methods. The background, 
structure and operation of CONTACT were described in full, 
together with a description of 'some of the problems 
encountered in its operation. 
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Mr M Head (HSIT. Aberdeen) in reply to'Miss Hustwit's PaP« 
emphasised the value of personal contact in ^formation transfer 
This point was also made by Professor Ely, and identified in the TASK 
result Several. doubts remained, however, on the basic structure 
of 'the informatibn held, and therefore of the efficiency of 
search strategies, in terms of controlled vocabulary, ^ 
definition of subject coverage and identification of 

user group.' . . , 

Discussion centred very much on practical operating details 
of the system described, and'as such indicated a great 
deal of interest being shown in the concept of a personalised 
• information system. Evaluation of the system by user 
feedback was considered by some participants to be 
a point which had been underestimated as a controlling . - . 
factor in the development of the system. 

Working Information Systems: 

Two Working Information SysAs were described by - 

the next two speakers MrWJl Davies and Dr D Bligh 

(the latter paper presented by Miss J Claridge, m the absence 

of Dr Bligh) . , 

QUERY, developed at Hertfordshire County Programmed - 1 . 
Learning Centre, as described by Mr Davies, is a 
Sstem^designed for classroom teachers. Mr Davies emphasised . 
that with. a computer-based system, a clear layout of 
complex information is most important for the user . 
He pointed out various limitations of the present QUERY 
system, luch as seaQential searching, which makes response 
times relatively slow, and various restrictions on record 
length and layout. Systems designed for microcomputers > 
tend 'to suffer from similar restrictions. There w 
apparently-a new, improved QUERY in preparation, although 
STye? available. It will still essentially be a locally 
^ based system, since Mr Davies believes that individuals or 
groups may usefully develop their own data retrieval • 
systems with their own local' idiosyncracies .in preference 
or in addition to using national systems. 

This last point was taken up by Mr L McMorran of Aberdeen • • 
College of Education Library, in reply. ' Mr McMorran 
emphasised the value of local systems for local needs, and 
specifically for changing needs, but pointed out at the 
IS time that many of the advantages of sophisticated search , 
strategy, and rapid access and interaction for users, at 
prelent taken for granted in large systems, are not available 
to small systems. There is a danger Of producing a 
" multiplicity of isolated systems using different programs 
becoming incompatible either with each other or with a . 
national system. Systems like ERIC, while perhaps not ideal . 
for local needs do -at least have the/advantage of a 
carefully constructed thesaurus of controlled vocabulary • 
terms and relationships. As Mr McMorran indicated, terms with 
vague and ambiguous, meaning ar* not uncommon m the free 
language of education or educational technology. 
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TEARS, developed at Exeter University (The Exeter Abstract 
Reference System) is, like QUERY, essentially a local system, 

'for higher education. Three problem areas in hiqher education 
were identified by Dr Bligh as areas in which an information 
retrieval system could play a Part: the system could identify 
relevance i.e. delineate the field; it could establish " 
authority for included items; and it may direct or assist in 
mapping the research field. The implications of an information 
retrieval system can be much wider than' the relaying of * 
references, and supplementary ^services such as an 'interest 
bank' of research or interest profiles could act not only as 
a search or alerting service, but, as an exchange arena % for 
teacher^ with similar, interests. In the paper, Dr Bligh described 
operation of -TEARS and the type of recdrd it contains. 

, Complex logic is used in search strategy, but questions remain 
relating to the amount of information to include, from what 
sources, and overall' evaluative questions *bf cost and charging/ 
User consultation and user evaluation were also 
mentioned, and the importance of personal contact *to user 
satisfaction, 

. In reply, 'Mr Martyfi Roebuck of the Scottish Education Department 
endorsed the belief that a system to be used must.be accessible, 
and must be needed*. In practical terms the cost of a system 
must compare favourably with other means of retrieving information 
perhaps more familiar to the user. Problems of recall or precision, 
and of /scale, were referred to briefly as worthy of consideration 
in designing systems. 1 Doubt was cast on the idea of having 
a universally acceptable thesaurus or map, since^from experience 
Mr Roebuck had found that the self-correcting hierarchy or concept 
map was perhaps more flexible. 




Qualified Citation Indexing: 

The concept of citation indexing may be relatively new to the t 
field of educational technology, but it is well established 
in other fields. Science Citation Index r Social Sciences 
Citation' Index and Arts and Humanities Citation Indetf, all 
published by .-the American Institute for Scientific Information 
\(ISI), each run to several large volumes per year of very small 
print. The theme of the papGr on qualified citation indexing 
presented by Mrs E Duncan (RGIT) was that citation indexing 
techniques are particularly appropriate for educational 
technology, because of the wide spread of subject interest ^ 
the difficulty of identifying a user population and the 
ambiguities of an international terminology. A qualified 
citation index, in which relationships between punished * 
works would be specified anc3 described by a qualifying .term 
or phrase is being developed in prototype on a 
computer at RGIT. Phrases such as 'similar research' 
or 'methodology' indicate the context in which a 
reference has been auoted.. The project ^sf sponsored by the 
Scottish Education Department and run joihtly by Dr R McAleese 
of the University of Aberdeen and Mr F D Anderson \* # 
of RGIT. 

Mr L Conbett of Stirling University Library, 'in .reply, 
^emphasised the size of files necessar jly created'by citation 
"indexing - manual handling is impractical, and out ut 
includes much 'noise 1 . Qualifying of citations therefore 
would a£>pear to be helpful. Other techniques, such as 
'cycling' have been used by ISI for similar purposes . . • 
Difficulties* remaininq for the project, and for citation 
indexing of educational technology, in Mr Corbett's view 
are those'of subject definition, and of understanding of the 
reasons an author has had for quoting a particular reference - 
- they are not .always clear or meaningful! > 
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WORKING GROUPS? 



Participants were asked to join one of three groups given the 
remits shown in Figure 2. "The overall aim of the 'Working 
Groups' was to identify ways in which information needs may 
bes€ be fulfilled in the three areas specified. 

.Group 1 discussions centred on the relative importance to 
users of centralization of information resources, or level 
of support. Questions aised by' the Group were whether 
centralization implies control, how immed ate are most 
information needs, and how difficult is it to identify 
the searcher' and thus match needs to sources. Results' 
presented visually below. (Figures 3 and 4) 

Group 2 produced a visual map of the topic 'independent 
study 1 * (Figure 5) with indications of relationships between the 
concepts displayed. Comparisons with construction of 
a linear thesaurus were made\and ideas on variation in 
interpretation of 'level 1 , 'timeliness' and 'value' 
discussed. It was felt that the graphical technique could 
be applied very usefully in other subject areas as an aid 
in identifying relationships between parts of subjects and 
evaluation of the importance of some subjects in relation 
to others* 

Group 3 concluded that citation indexing could be a helpful 
approach to an' information .query, since often the query 
begins with knowing people or authors working in ar^area. 
It could be. used as a way of seeing how ideas recur in- 
the literature and could be helpful in solving unformulated 
queries* Again discussion took place on who the user 
community will be. A sample search on SOUCIT, the 
system being developed at RGIT, is included in Chapter 6, 
following the paper on Qualified Citation Indexing. 



Presentations from Groups were made by the three appointed 
» reporters and the issues discussed by all participants. 
Final issues seen to be ijtjgortant by those present are, 
summarised below. 

$ 
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» GROUP 1: ■ . ^ 

This group will consider -whether existing resources in 
.educational technology m^et the' needs of individual users 

.- identify the range of .resources available 

- classify the resources 'into a framework that places 
similar resources together e.g^. books, journals . . 

- agree on reasons- where existing resources don't seem 
to meet the conditions you. expect 

, - report on some suggestions as to how resources might 
be made more useful ^ ' 



• GROUP 2 : 

t • 

- identi f y > -ap-iippor t ant concept, in educational' technology 

- list 11 related concepts 

- construct a 'map' of the concept identified -,i . e . » 
indicate the Relationships between the prjLme concept 
ana tne related concepts . 5 

- report on how the mapping technique may be usefully 
applied to other areas 



• GROUP 3 :• 



ita/fcion 

- list the ways in which the retrieval . system can help 



' .- undertake a citation search on the QC'IP citation data 
base 
list 

the 'user , i ^ 

-.- repo rt on the ways in fyhidh the system could be 



improved 



A- 



Figure 2 : Working Groups - Remits , 
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WORKING GROUP 1 - RESULTS 



, Ephemera 




Newsletters 



Abstracting 



Figure 3 ; Printed, or 'media-based ' resources 
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WORKING GROUP 2 - RESULTS 



V 



SELF-STUDY 
TECHNIQUES 




Type of relationship identified - 
synonyitous 



} 



s e que nt ial /te mpor a 1 

' generality y\ . 

hierarchic m ^^ZZ^^ 



prerequisites | | | j 

} 

logical entailment 



lateral 



o 



purpose 




Figure 5 



A^ 



' Map' of the concept 'independent study ' 



(♦This is an unedited version of th.e transparency presented 
by »the Group) 
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IS&JES: % # 

Prom the variety of presentations during the day - the 
papers, the TASKr--^he--Wr # k ing Groups and the discussions - 
a number of tissues emerged. These issues are important 
as they indicate the problematic natuire of educational 
information retrieval/ and perhaps provide indcators to the 
, future. In summary the issues were: 

1. WHO is the searcher? 'It is important to begin with the 
SEARCHER - after ,all, 'no system -would be needed if there 
was no enquirer. Is it an experienced research worker', 
or is it a student learning the subject and the 
. languaqe of the subject? Different searchers need 
different facilities and different types of information. 

%2. LEVEL OP SUPPORT - Where is the information to be located? 
Will the focus be local or national <?r international? 
A local information retrieval system has very different 
characteristics from those of 'an international facility. 
A good comparison to make this point would be the- Exeter 
system, TEARS, (described by Bligh) , and ERIC. * ^ 

3. IMMEDIACY w How soon does the user need the information? 
Will tomorrow be too late? What do we know about the 
'intellectual windows' where researchers and thinkers 

need to Jcqpw that missing piece of information 
IMMEDIATELY>be£ore they can continue -with their creative 
thinking? Does one need on-line search facilities or 
tyill a postal retrieval system meet £he needs of- mpst 
users? s ; 

4. PRESENTATION - Is it good enough to see the information 
on a VDU screen, or does the searcher need the hard copy- * 
of printed text? The ergoncmic, -as well as social and 
psychological issues with regard to user interaction 
with data are far from clear. Can 'Systems be made 
user-friendly for the interactive and mediated searcher? 

5. Is there a need for INTERMEDIARIES? 'Is* there a role 
for the information scientist to counsel the searcher 
and identify profiles and user needs? Who is best 
qualified to make the intervention? Librarians and information 
scientists, or subject specialists? 

6. Are there LIMITS TO TECHNOLOGY? Can the technological 
answers provided by large computer-based systems get in 
the way of the searcher? Are we in danger of looking for 
elegant technical solutions when the searcher needs the • • 
feel of a few. back copies of a journal, or an hour browsing 
from shelf, to shelf in a library? 
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The Symposium did more to raise to the surface some of the 
issues than to provide solutions* Perhaps the model of the 
symposium, in which some of the thinking was left to the. 
participants, is one which the information scientist ^ s 
must consider when designing information systems* An ideal 
solution is often one where some of the decisions are 
left to the searcher and where most of the hard drudgery 
of searching is removed* By the time the, Second Information 
Retrieval Symposium is held in 1982, some further issues 
will emerge and perhaps some clarification may be made on 
* the information needs of the educational technologists. 
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CHAPTER 2 

Educational Technologists as Consumers of Educational Information 
. " Donald P. Ely 

Syracuse University 



Information is a basic commodity in the world of the educational 
technologist. It is needed for clients and colleagues as well as for 
personal use. As in other fields, it is available 'in greater quantity 
and in more locations than ever before* The saturation point may have 
been reached and 'exceeded. Like other professions, educational 
technology is suffering frofii information overload. 

The overload phenomenon createl a situation in which the educational 
technologist can no 'longer carry all the needed information in his or 
her head and must. adopt infc>rmatiOTi seeking .strategies which help to 
obtain needed information. These information seeking, strategies become 
increasingly important as more and more information becomes available 
and as technological systems contribute to *the professional* s ability 
to cope with the mass of information* 

Information used by educational technologists is not produced in 
any systematic fashion. New journals appear with increasing frequency; 
publishers release more titles in the field each year; reports from 
goverhmental agencies and professional, organizations have reached the 
proliferation stage; and still it goes on. Yearbooks, dictionaries, 
encyclopedias, and books of readings are signs of maturity .in the 
profession but, they also add to the volume of information with which 
the consciencious educational technologist has to cope. ' ^ 

Information. Coping Strategies 

What arfe the coping strategies? Informal Conversations with 
professional colleagues have revealed a wide range of coping , * - 
behaviours. For example, ^ . t 

- One educational technologist has- narrowed his professional 
reading' to those journals and reports which encompass his 
specialized area of interest. He has "given up" trying to 
relate his specialty to the larger context of the field. 

- Another person scans journgls^ reports, books, and other \ 
literature to provide an awareness of current developments 
but he does not read extensively or deeply in any one area. 
If. he needs information, he deperids upon recall to retrieve 
the specifip item he requires. 

- Still aAother person holds on to -familiar information which . 
she has used in the past and continues ~to use it. She reads 

-newsletters and state-of-the-art articles in journals to keep 
current. 

There probably are as many strategies as there are people in the 
field. Each professional seems to have his or her own profile for 
information seeking. ' , 

& J 
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Major Information Sources 



Several attempts to c6ntrol the information explosion have been 
mads. 'Notable among these attempts, is the ERIC system in the United 
States. The 'Education Resources* Information Center (ERIC) system was 
established in 1967 to capture the "fugitive" .literature in Education: 
reports, speeches, curriculum guides, conference proceedings and other 
ephemeral materials which do not usually gfct into the mainstream of 
educational information. The .rationale for Inhering, storing, and 
making available fugitive information is that literature which is 
published by commercial houses and major organizations and agencies is 
gener&lly^known and easily available. The more' obscure literature, 
much of which would? "be highly valuable to some users, is nbt known 
and is rarely accessible. In 1 969 , ERIC began to index journal articles. 
The total number of citations for documents and journals at the end of 
1980 was approximately 420,000 items. Thus ERIC began with separate 
clearinghouses responsible for specialized fields. In educational 
technology, the original clearinghouse was named "educational media 
" and technology". It later merged witl\ "library and information 'science" 
to become "information resources", its present designation. ERIC has 
become one major source of information about educational technology. 

• Other efforts to manage information have been brought about by 
synthesis volumes such as the Educational Media Yearbook (Brown, 1980) , 
the Encyclopedia of Educational Media Communications and Technology 
f TTnwin and McAleese, 1978) and the International Yearbook of 
Instructional and Educational Technology (Howe, 'i960) . Each of these 
volumes offers one source of comprehensive information about the field. 

To provide timely news about current developments in the field, 
several newsletters seem to serve the purpose: Educational Technology- 
News (from AETT) , eict (from AECT) , and dozens of local update 
newsletters which axe brief, inexpensively printed, and devoted more 
to announcements and current activities than to substantive matters. 

There are several major publishers which emphasize educational 
technology: the Association for Educational Communications and 
Technology y the Council for Educational Technology of the United 
Kingdom, Educational Technology Publications, and Ko^an-Page. Of 
course there axe many more publishers of books in the field of* 
educational technology, but these seem to have the most extensive 
catalogs and probably 'represent 'the majority of sales in the field. 



Information Needs of Educational Technologists 1 

Given the multifarious sources of information in the field, where* 
does the educational technologist go for information? The question 
seems straightf orward enough but it is necessary to narrow it still ^ 
further. * % 

mm What type of educational technologist? the researcher? 

the administrator? the professor of educational technology 
-> in a tertiary institution? the producer of materials? the 
instructional developer? tf 

- What is the specific purpose of seeking information? to write 
,a paper? ^to prepare a speech? to justify a budget? to answer 
a colleague^ question? to become better infefrmed? 




- What p esourcles beyond published, sources might be used? personal * 
files? colleagues? audiovisual materials? 

. # 

These question's were inferred in a paper presented by Ely at the 
ETIC 79 conference at Sheffield: 

Improvements in the current information environment 
f 01^ educational technologists should be preceeded 
by a study of the information needs of the population. 
Input criteria and output potentials should .be .altered 
^accordingly. Existing systems should be modified 
rather than to .oreate new systems, (p. 31 5) • 



Pilot Study of Information Heeds 

In* an attempt to focus on the information' -eking behaviours of 
the educational technology population, a pilot study was planned and 
conducted among 42 chairpersons of doctoral programs in educational 
media and technology in the United States listed in the Educational 
M^dia Yearbook 1980 (Brown, 1980). Faculty members from four large 
academic programs in educational technology were also part 6 of the 
test population. The total 'number of individuals quarried was 60; 
the response £ate was 



Individuals who teach educational technology were selected as the 
first g^oup for study since they are likely to use many and diverse 
sources of information. Presumably there is a higher percentage of 
authors among this group than among other groups within the^field. 
The need for information is, therefore, highly specific, information 
is -probably more important for this group than to any other identifiable 
^roup within the field. Thq pilot group could serve as a prototype 
for other identifiable groups within the field, i.e., educational 
technology administrators; instructional developers; 'and media 
designers and producers. ( . . . 

The study probed fpur questions: * * 

1. What sources are used for current awareness of -new-develop- 
ments in the field? 

/ * 

2. What sources ar,e used for* six specific purposes: 

' ' - 1 - * S 

a # Preparation for teaching? 

b. Writing a manuscript? 

»*c. Preparing a speech? " 

d. Responding to colleagues* questions? 

e. t3ase studies? 

'f. Budget justificalfcon? 

3/ What professional journals are read and how are they used? 
* ■ .' . 

4. What are tfife preferred formats for aWa»eness information and ^ 

^ research information? 
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It was thought that answers to these questions would provide 
^sufficient indicators to describe the information seeking behaviours 
of this group of educational technologists. It also would help to 
develop strategies for targeting information to this audlenpe. What ^ 
was discovered? . * * 

Findings of the Study - Sources Used ^ 

Individuals who head educational technology departments 'and teach 
in tertiary institutions use prc*fessional, meeting^ most frequently for 
current awareness about, new developments in the field. The. meeting 
most frequently mentioned was AECT - the Association for Educational # 
Communications and Technolipgy, followed closely by < AERA - the American 
Educational Research Association. Informal contacts was the $ext most 
frequent category for current awareness. Itotlr face-to-f ^ce and 
telephone contacts were about the same. Relatively higfr,Jbut third in 
frequency, were newsletters with tfte ERIC/lR Update! Wid the 'Chronicle . 
■ of Higher Education {being the most frequently mentioned i^ems. The* 
individuals who responded to the survey seem to use -all* three, modes 3 .« 
for current awareness with a relatively high frequency. 

, In the report ty Gilbert ( 1979) individuals From ^ higher education 
institutions* in the U.K. were interviewed about the use of educational 
technology research. In response to one question,, "How do you find 
' research results?^ Gilbert repor/ts that the responses "might, have < 
been influenced by the facilities available, personal' motives for 
seeking such results and perhaps even a range 6f developed capacity 
for the use of resources. Ke indicates that the following aijproac^ies 
are widely used: ^ * / 

1. Personal literature surveys,' ... To the so called f .ed. tech.* ( 
literature is Bfeing increasingly added an examination of those 
professional discipline journals in which relevant articles 
appear. * . \ ^ ^ 

/, 2. Abstracting services ... However, no comprehensive abstracting 
service is yet available in the absence^ of a commonly agreed 
categorization system. 

4 • V 

3 # Personal discussion. This takes place eit^ej? through an 
informal network ... or at conferences".^ 

Other approaches include the use of bibliographies ^on specific 
areas, books, and ERIC (which, according to Gilbert, is little known 
or used) . 



In attempting to dicovSr primary sources Qf information, six 
common uses of information were determined: \ 



— preparing"f or teaching 

— writing a manuscript for publication 

— preparing a speech t 

— responding to colleagues' questions 



\ 



— locating case studies 
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Use of Professional Journals 
: r 

Since professional journals are considered to be a prime source of 
current research and development information, it. seemed useful to inquire 
into which journals were read, the nature of their use and the extent 
to which they are retained for future Use. Table f- presents the findings 
in regard to journal use. ■ 



Educational Communications and Technology Journal 

Instructional Innovator / 

Educational Technology ^ 

Journal of Instructional Develppment ' . 

i 

/ 
/ 

THE Journal 

Public Telecommunications' Review 

British Journal c-f Educational Technology 

School Ife^iia Quaker ly " 

Instructional Science 



Wo 

31/0 
31/0 

29* 



Table 2 Journals- Read by Academic 
Educational Technologists (n = 42) 



Respondents ware given the option to indicate the extent of use: 
' ".read thoroughly," "read selectively," or "scan". In every instance, 
more persons "read selectively" than "read thoroughly", or "scan"* 
For the' four most frequently' read journals, an average' of 17$ of the 
rea£e: \ jead the journals thoroughly. When asked to indicaie the . 
disposition of eacfc. journal, "save," "pass on," ot "throw out," the 
most frequently read^ournals were the ones most frequently saved 
^ and very few journals were reported to be "thrown out . 

jr 

> Other journals listed had fewer than 12 responses: Journal of . 

. „ Computer Based Instruction and the APLET Journal . Thirty-one 
additional joufnals were named by the respondents.- The most 
frequently mentioned were Review of Educational Research ^ AERA 
Journal i Media and Methods , PHi Delta Kappan , Educational Leadership , 
and NSPI Journal - a]J of which had four to seven nominations., 

In a study done by the Associations f or Educational Communi cat ions 
•and Technology in 1977, 2,612 respondent 8^36$ of whom were from higher 
education institutions, Audiovisual Instruction (now Instructional 
Innovator) was the most frequently read joupnaj (89$), Media and 
^ Methods, was second ( 56$) and Educational Technology was- third ( 44%) . 
q The results are reported on the -total population, not just the higher 
ERJC edupation portion. 



Another study corroborates the findings of this 'survey. * Moore * 
( 1981) asked 350 professionals who were members of the Association 
for Educational Communication^ and Technology (AECT) in the Unit&< 
States to rank order (from a list of 11 professional journals) 
" ... the <three you consider to be most influential within the field 
of instructional technology". Individuals were told that they could 
name journals not listed. The 142 respondents, 33 of whom were* 
college professors in the field,' rank ordered! the journals as follows: 
Instructional Innovator ( 75. 4%) , Educational Communications .and 
Technology Journal ( 45* W > Educational Technology ( 44»4%) \ _Media and - 
Methods* ( 44. 1$) ♦ Journal of Instructional Development ( 26.8$) and 
Educational Broadcasting (9.[ 



The saipe population was asked to rank order "... from the • / 

•following list of professional journals the three*you consider to 
be the most scholarly within the field of instructional technology". 
Journals could be added to the list, (in fact, a total of 44. journals 
were mentioned on the. forms) • * s " 

Educational Communications and Technology Journal 78.5$ 

' Jotirnal of Instructional Development 57«0$ • 

Educational Technology * ^ '38.5$ ^ 

British Journal of Educational Technology 19* 3$ 

Instructional Innovatpr * , 18-. 5% 

International Journal of Instructional Media 14*8$ 

It seems reasonable to cpnclude that journals considered to be 
* ? most influential" in the field and those which are considered to be 
"most scholarly" by a general, population of educational technologists , 
are ^fchose— journals which are -read-most- frequently by. a5_ademic^ducationaL_ 
technologists, except for -two journals considered to be influential, 
Educational ■ Broadcasting and Media and Methods , and one oonsidered to 
be scholarly, the International Journal of Instructional Media . 



Preferred Formats , 

A final question had- to do with the preferred "format for awareness 
information and research information. For awareness, ( 64$ of the 
respondents preferred professional journals and 2>&?o preferred specialized 
newsletter^,* For research information, t abstracts and journal articles f 
were equal jhalf Prefer abstracts, half prefer articles.- No one appeared 
to want^ompiete research , reports. "^-^ 9 



Limitations of the Study 

A3 with any* pilot study, certain cautions must be stated. The . 
population is Ignited to individuals who^teach educational technology 
in tertiary institutions. Their needs for information and access to- 
• it are probably quite different fronf ot^her populations within the field, 
§ # g. instructional developers and managers of media "programs. The 
population studied is more, likely to seek research-based information 
and to read journals which report research. 



Findings and Generalizations 

With these preliminary findings in mind, it would be useful to test 
them againstia list of generalizations about information needs in a 
number of fields derived f*rom more than /I00 studies and summaries of 
studies (Fabisoff and Ely, 1976) in the literature on user ' information 
needs. The l^st.of generalizations is highly tentative and in need of 
further corroboration. They were developed on-the basis that some 
evidence, however meager, is better than none at all. Therefore, 
'caution must be exercised in using the list. 

. 1. People tend to se-,k out information which is most accessible . 
This study confirmed this generalization. In every instance of 
- information seeking, personal files were reported to be the 
• first source used. For service activities such as locating 
case studies or budget justification, face-to-face and 
telephone contacts followed personal files in frequency* For 
academic pursuits such as preparation of a manuscript or 
speech, the fc university library was used. 

2. People tend to follow habitual patterns when seeding information. 
It seems obvious that one would first seek our information in , j 
a personal file. This habit seems to dominate even -when, the' 

'library is a more obvious source to locate information for / 
academic purposes. It may also be a function of distance- ana 
ease of access. 

3. Face-to-face communication is a, primary source of information . 
Studies/ of professionals and researchers in a variety of 'fields 
indicate that they often contact other professionals and re- 
searchers they know have the^ information required at the moment. 
In this pilot study, face-t<>-f ace -and telephone sources ranked * 
just belpw personal files for service-typejlnf ormation .and 
so:r.r*v]"j<ii lower for academic information. 

4 # The information needs of the individual change at different 
career stages and with changes of projects . ♦ 
"It is, perhaps, self-evident that informaton needs wilj. vary 
not only according to th^siibject interest of the users, but 
likewise in relation to^|^ype of activity in which they 
are engaged". ( ftees^iflji this pilot study, it appears 

as if t^ie strategiea^ised for obtaining academic information 
vary with those in seeking .service-oriented .information. 
There are probably very few needs which recur regularly which 
individuals do not form a strategy for meeting. The changes- 
of information needs over' time differ in substance as well as \ 
in spurce. 

Guidelines for Designers of Information Systems for Educational Technologists 

While it is risky to use the findings of one small pilot study to 
create information dissemination strategies for an entire field, it seems 
worthwhile to propose some tentative guidelines which build on similar 
studies performed for other professional fields. The tentative nature 
of these reQQnjjnendations must be underscored. Alan Rees* Observation 
(1963) that the information retrieval field has been plagued* f or many 
years by busy people spending huge sums of monsy, designing - or attempting 
to design - phahtom systems for nonexistent people in hypothetical 
situations with unknown needs must be borne mind. 



A sm^ll but ardent core of researchers. has addressed the infbrmatibn 
needs question. (Mensjel, 19^0; Brittaip, 1970; Line, 1971; Dervi^t/^73) 
^Their investigations produced results. The users were identified ar 
their needs characterized. Re^ga^ess of ■ dicipline, occupation, or 
level of education cf the user, certain patterns of needs ^commor 
began to appear # ^ These patterns had implications fdir^ie ijngWvennerft 
of and development of new information systems". ( Pab\sxi|^^d/ Ely , 1*976) 
Specific guidelines for educational technology information systems -are 
drawn from e larger list of guidelines. > - ^ 

1 . Identify the specific information the User actually needs or 
requires for what he/she is doing . ^~ " 
To design infornicCiS < r. systems without knowing the information 
needs of users seems to belabor the obvious.' However, this 
principle is often overlooked. Brittain ( 1 970) observes that 

•"There is... a movement toward a receiver 1 controlled system 
and movement away from a source ✓controlled system. The 
y attention to user-oriented information systems is only one • 
example of a growing phenomenon". Computer based information 
systems make the user orientation^more feasible today than 
ever before. ERIC is a good example of such a system. 

x 

2. - Identify th| user in relation to his/her discipline or 

environment . 

♦ * It is not enough for the informatiQn systems designer*to know 

that an individual is an educatior^l technologist. Subgroups ^ 
within the field must be Mentified since each one requires 
a 'different type of information to satisfy* his/jier needs*. Rep- t 
lications of the pilot study with different groups of educational 
{ technologists would probably yield quite different information 
seeking patterns. % 

3. Information-should be provided in a form suitable f orbits N 
effective use . 

A well-designed system should be tailored to suLt the require- t 
ments of its users and should not .place upon the user an wndue* 
burden of mastering a new discipline. ^ The population in the > 
pilot study, said they wanted awareness information in 
professional journals and research information in abstracts' 
and in journal articles. Journal editors take note! ERIC 
provides information by title and abstract and complete 
reports. Very few academic educatiorikl technologists want the . ' 
complete report. * 

4. Information should be stored in such a way that ,it is n^fc only - 
available but easily accessible . > * ' / m 

" The system should be as simple to use and as accessible as * . 

possible. The principle of least energy and Effort applies 

here; Dervin ( 19?3) identified five drmens^ons of acce^si*- 
* bilityt > 

* . ■ . %: ; ' > 

- Societal accessibility - mt*& be Mailable' in. the social^ 

* - * system. 

- Institutional accessibility - must be capable and willing . 

\ .to deliver needed information. 

•* . * ♦ , * 

- Physical accessibility - individual mu^t be atile to 

make contact wrth the source. 



- justifying a budget 

FivG common sources' of information were listed: 

- personal library or files 

_ face-to-face contact with a colleague 

- telephone-contact with a colleague 

- department or colleague 1 s library 

- university library , 

Respondents were asleep to' indicate, in priority order, which sources 
would likely be used for each purgpse,- Table 1 reports the results based 
on 42 replies using the modal response. 



Per s onal P ac e-t o- 



Telephone Department University 



Piles 



Pace Contact Contact 



or , Library 

Colleague' s 
Library 1 



Preparation 
for, Teaching 

Writing a 
Manuscript 

Preparing 
-a Speech 

Responding^ . 
to Questions 

Case 
Studies 

Budget 

Justification 



3 

3,4 



V 



4,5 
5 
5 



3 



4,5 



5 
5 



Table 1 Information Sources Used by Academic Educational 
Technologist for Six Purposes 



With this particular audience, it seems quite clear that a person's 
""own- library and file is a prime information source" for virtually every 
purcose. For more scholarly activities, such as teav^iing, writing, and 
' speech-spreparation, the individual turns to the university librajr while 
for more applied matters jteuch as question answering, case studieB and - 
budget justification, person-to-person contact-s and telephone contacts 
seem to be major sources. The use of a department or colleague' s -library 
" does not seem to be a dominant source for any purpose. 
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- Psychological accessibility - individual must be willing to 

approach appropriate' sources 
and believe that answers can be 
obtained. 

- Intellectual accessiblity * - individual must have ability 
, * % and training' to acquire and 

'process needed information. 

5. Since oral communication is an important feature of gathering . 
information, the system should devise ways for facilitating the 
dissemination of such information . 

Informal channels of communication referred to as invisible colleges 
are frequently the most significant means of obtaining information.. 
The pilot study showed that such sources are often preferred over 
more foritial sources. The fact that professionals consider profes- 
sional meetings as a prime source of- awareness information is 
further confirmation of this source being a vital element of almost 
any information-seeking activity. It would seem, therefore, that 
directories of programmes and personnel and professional meetings 
ought to be planned.and promoted as information sources. The 
professional educational technology associations (such as AETT and 
AECT) provide a major information system incorporating opportunities 
for oral communication ( through annual meetings) along with 
publications, newsletters and adjunct nonperiodical publications. 
1 It probably would be difficult survive in today* s information 
environment without an active professional association. 



Closing Words * 

I do not want to make more out of a pilot study than it deserves. 
This little effort is' but one more indicator' of information usei^iy 
a sample of one professional population. It can be added to the 
cluster of studies of the same genre. 

What I am attempting to show here is the variety of information 
seeking strategies used even within one small group. I am attempting 
to point £ut the need for corordination ajnong the wide variety of ^ 
information source's. Increasing the number of information sources 
does not necessarily mean better information. But better understanding 
of who the audiences are and what information needs they possess 
should, help to develop more efficient use of the information sources 
currently available'. ' Just as instructional protocols can be 
systematically developed to achieve predetermined objectives, so 
can information sources be configured to meet specific information 
needs of specialists. Just as front-end analysis and formative 
evaluation have been integral t^ well-designed instruction, so should 
information needs and information seeking behaviors be factored into 
the design of information systems for educational technplogists. 

^ I should clarify the reason for purposive .design of information 
systems. It is not to create one ideal system for all to use but, 
rather, to identify, co-ordinate^ and communicate the information 
4 options available and to facilitate the availability and access to 

the individual user. Beyond availability and access is the need 
for assistance to help the user orchestrate the multifarious resources 
to meet his/her specific information need. Mere availability will not 
• suffice if the individual does not know how to manipulate the sources 

O to achieve personal objectives. 
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The next stjap seems to be the development .of a. pilot information 
system whTchtakes Into account the user variables. < ■ 
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N. Rushby, CEDAR, Imperial College, 



Questions arising from the paper given by Professor Ely cafh be 
summarised as follows: 

1. Is there a conflict in journals between - 



2. Simi 
a 
b 
c 
'd 



what tjie authors want to write? 
what the editor wants to publish? 
what the reader wants to read? 

what the reader should read r and if this, who says so? 

c 

arly in information services - is there a conflict between 

what information is available? ' ^ 

what we can include (in terms of available resources)? 

what users perceive they want? 

what users really want? 
» 



3. What about subject teachers who are not specialists in information 



seeking? 



> 



Is there confusion in some minds between uses and needs? 
To- what extent are users 1 needs shaped by what is available? 



o 

ERIC 



30 



♦ CHAPTER 3 
Information Services in Practice 

Jane Hustwit , Council for Educational Technology, UK 
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The Council for Educational Technology, sincejlts creation, in 1973* has 
always taken an active interest in the proBWms surrounding and blocking, 
access to information - information which may be of use or value to 
educational .technology practitioners. For example, the publications of 
Gilberts inve^iagation into the use of research findings by ^practi- 
tioners of educational technology (1977) and the complementary work 
by Clarke* and Dunn from Glasgow, also published in 1977. 

During* the last couple of years we have also become more aware of the 
number 6f Council activities which have a key information element in 
their successful design* Eie major development project on open learn- 
ing, systems has both an information service and a newsletter, created 
specifically to inform people about this work. Those working on the 
regional coordination of .educational technology arrangements are also - 
producing a newsletter to. ensure that their constituency is^informed. 
In addition, over recent years, there has been a tendency to focus on 
the difficulties often experienced when attempting /to provide the 
educational practitioner with a means to locate information. Given 
the diversity of disciplines and subjects consittered^as comprising 
educational technology and the range of people interes^fed^in using 
educational technology to improve their teaching, this generates fairly 
sophisticated problems* 

The Council has recently been working in two broad areas - the develop- 
ment of experimental information services, coupled with a continual 
investigation of the application of new technologies to these "problems. 
It is the first tfrat I would like to discuss here. Before discussing 
CONTACT,., the Councils information service on teaching and learning 
methods, I would like to look briefly at the experience of our* inform- 
ation officer based at CEDAR, Imperial College, London, who was appointed 
in 1980 specifically to work on computer-based education.^ 

The CEDAR project was set up in, 1977 with the aim of encouraging and r 
facilitating the development and effective use of educational computing 
.both within and beyond' Imperial College. It benefitted considerably 
' . ^from access to the stock of computer based learning materials and 

information which had been collected by the National Development Pro- 
' gramme for Computer Assisted Learning (1972 - 77) • 

The information service, originally designed to serve the College, 
rapidly grew to provide a more general service for theeducational and 
training community. It now provides a variety of iriformation about 
computer based learning: 

* - general descriptions of computer assisted and cfomputer 
jnanaged learning 

/ 

- descriptions of specific educational computing projects 

- information about the -implications of computer based 
learning for curriculum design and assessment 

- information about the technology associated with computer 
based learning. 
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From 1978 onwards, the amount ',of information available through CEDAR 
grew substantially and the number of enquiries rose dramatically. 
At the same time, the Cqjuncil was acting as supplier of 1 aftercare* 
for NDPCAL. This involved providing an information service on computer 
based education, as well as other tasks. Here also the number of 
enquiries shot up and staff resources became overstretched. So it 
was agreed in late 1970 that the Council would* fund a full-time inform- 
ation officer to operate and develop the CH>AR service further. 

This person would be required to: 

answer external enquiries about the application of , 
computers t,o education with access to the CEDAR 
databases, bibliographic sources and records of 
experts; 

systematically develop the records of experts and their 
expertise; 

collect ^material for publication in CALNews. 

These activities would, it was agreed, all contribute towards the main- 
tenance of a central information referral ptfjnt for enquiries about the 
application of computers in education. 

I would like, at this stage, to point out the differences between this 
service and that offered by CONTACT. 

1) The bulk of the enquiries are received and answered by post 
and telephone, although the project does receive a number of 
visitors. 

2) There is an important collection of resources and materials 
to be used and maintained by the information officer. ' The 
college funds a database holdirig detail's of books, reports 
and papers on computer-based learning and related topics, 
and details of CHL packages available in the UK. The da tar- 
base can be searched on line for specific enquiries and it 
is also used to produce catalogues^ both in print and micro- 
fiche form. * ' 

The information officer also has access to the CEDAR collection 
of publications, housed within the main college library. 
These resources have grown in the past year - the CAL Package 
Index has been increased from 400 to 500 packages, find the CAL 
Bibliography from 1400 to 3000 entries. A People find Projects 
Index has been set up recently, for instance files on Primary 
Schools, Handicapped People, English Inching have been 
created, according to demand. 

Ideally she should also be able to demonstrate computer 
assisted learning to visitors, but there are$ as yet, 
insufficient resotifrces available. s 

.3) The information officer has a responsibility for providing, 

the basic copy for CALNews. This free newsletter is circulated, 




three times each year, to all types of educational 
and training institution, all over the world, and 
appears to be much appreciated. 4 ^ 

It should be emphasised that our work with* CEDAR was developed as a 
response to a clear and pressing need,, experienced. very acutely by 
staff already working in the area. CONTACTS creation was\quite 
different as the project , developed out of discussions amongst staff 
atxCET which were, in turn, provoked by research and ideas prominent 
in 1O76. The main theoretical concern was to improve the dissemii>- 
atioh of information within the education and training system, 
particularly information about practical applications of educational 
technology.- It was apparent that much wasteful duplication of 
effort could be avoided if a practitioner wishing to introduce a 
new/teaching or learning method in Jiis or her institution could be 
put/ directly in contact with someone with experience and expertise in 
th/t area. 

These discussions coincided with the final report of the Nuffield Group 
for .Research and Innovation in Higher Education. This pu$ forward the 
view that innovation in higher education depended more on contact between 
people than on elaborate documentation* 

"The exchange of information works best, it £eems, if it is personal. 
A lecturer in one polytechnic is more likely to contact a lecturer in 
another polytechnic about a new,.... .development (if he believes it 
might be relevant to him) than to read a lengthy description of the 
scheme. In other words, contacts between people are at least as 
important as documentation, and this country is small enough to make 
the former feasible and natural .... It is tempting in the present 
economics climate, to think o£ waste only in the more obvious, material 
ferms: but there is little point in saving time and taoney if at the 
same time one is squandering talent or throwing away good ideas". 3 

At the same time, the .Council became aware of NEXUS, an information 
system then operating in the USA under the auspices of the American 
Association for Higher Education. -This was a telephone operated 
1 people bank 1 , a linking service connecting enquirers with knowledge- 
able practitioners in the fifeld of enquiry. Anyone connected with 
higher education embarking on a new project or development could dial 
NEXUS and expect to be put in touch with someone who had relevant 

experience in that field^ m * 

• » 

Eventually it was agreed that an information service should be designed 
to share and utilise the practical knowledge and experience of many 
.educationalists who were,' at that stage, unable to disseminate it in 
any way 'themselves. Again the Nuffield Group's report stimulated the 
initial thinking "some of the best suggestion for avoiding waste are . • • 
local ones, Jborn of local situations and local possibilities. * The man 
or woman on the job knows more about it than anyone else, and hence 
often understands* best where useful changes can be made. (Therefore 
we wished) to fodus attention on such local ingenuity and to emphasise 
its importance" • 

CONTACT was set as a telephone service, designed to provide enquirers 
in higher education with information on teaching and learning methods, 
when they need it. It is a referral service, the information provided 
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being the names, addresses- and phone numbers of people with experience 
and expertise in the field of the enquiry. 

Between 1.00 p.m. and 4*30 p.m. every weekday, the CONTACT information 
officer is available to receive calls on the special CONTACT telephone 
number. After a preliminary discussion, in which the enquirer f s needs 
are carefully explored, a computer index is used to produce references 
to a number of record cards. These are theri consulted by the inform- 
ation officer, who selects the most appropriate and gives that info rm- 
atipri to the enquirer. Most enquiries are dealt with within 48 hoilrs, 
some can be answered at once. 

Some examples of CONTACT enquiries: 

- Is anyone using computers in biology teaching to illustrate 
points either in practicals or lectures? 

* - I need to .speak to someone with experience in the teaching 
of mathematics to groups of social scientists and account- 
ancy students with mixed academic backgrounds. 

- I want to know how I can assess student and staff reaction 
to the use of ETV in chemistry. 

- Is anyone in law using other than traditional teaching 
methods or using innovative assessment methods? 

*> How can I help a student with dyslexia? 

- Can you {ielp with problems of field courses - not 
necessarily in geography? 

- Study skills for, mature students - I T m having difficulty 
in getting them to open up in seminars. 

I will now look at the special features of the CONTACT system in more 
detail. 

1. Structure . 

The. basic concern, when creating the database, was to ensure that develop- 
mental work was located and the details stored. The aim was to develop 
a repository for contemporary information about work which may,, in fact, * 
never make the printed page - the Small localised pieces of work that 
the practitioner is too busy to write up.' 

•Construction of the database had to be undertaken immediately CONTACT 
was officially set up. A difficult task since it was quite impossible . 
to forecast the range or type of calls. The problem was approached 
in three complementary ways: 

1.1 through scanning' journals, periodicals, newspapers', bulletins, 
^reports of conf erences, workshops and seminars. 
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1.2 by fully utilising the "grapevine" 'thai always exists in v 
any particular area of interest, and by recording the new 

• information uncovered whilst answering a specific enquiry. 

1.3 by means of a questionnaire- type card, circulated throughout 
institutions of higher education. 

This was brief and to the" point, 1 It was mailed together with an 
information sheet- setting out CONTACTS aims and activities* This 
also doubled as publicity material. Hie cards were circulated by 
*link people 1 in every institutictn and were to be completed and 
returned to CONTACT. This system has now developed into a three-' r 
• stage annual mailing. ^ J// 

The success of an informatiop. retrieval system depends largely on the 
efficiency of the keyword system. In CONTACT'S first phase, 'keywords 
were created without 1i mixtions as each CONTACT; information card was 
received, or filled in from journal scanning since the area to be 
covered was,* to a large extent, unknown. This unrestricted use of 

" keywords was valuable in that it helped* to define the possible areas 
of importance in CONTACT information* From the first 801 information 

' cards, about 1200 activities cards were created, and these formed the 
basis for the subsequent keyword system. This method does also, of 
course, mean that the database is as cross-disciplinary as its users. 

A critical aspect of any keyword system is the kind of physical support 
system available. With CONTACT it was decided to use a computer for 
keyword searching, and fortunately discussions with the University 1 s 
Computer Unit resulted in the development of a simple interactive 
program, tailored exactly to CONTACT 1 s* needs. 

In addition to the alphabetic lists of keywords, a taxpnoray of key- ■ 
words has bepn created, to enable retrieval when the keyword was not 
known. This was created from the initial keywords. Several tax- 
onomies were considered and eventually the SRHE abstract inclex chosen 
as an initial classification. This taxonomy is regarded as a frame- 
work which can easily be modified as new developments occur. 

2. Updating the infonnation 

Not only does the CONTACT system incorporate new infonnation but, so as * 
to reflect the current state of a persons work and whereabouts, it must 
•also update old information. Here the use of an interactive computer 
file is infinitely preferable to the time-consuming labour-intensive 
retrieval of dozens of cards from a manual file, particularly as 
additions and deletions can be made- at the same time. 

CONTACT'S service is .tailored to the needs of each enquirer, for instance 
each card can be discussed with the enquirer and accepted, or rejected, 
extremely quickly, Repjlies are paced to the caller! s speed for, with 
CONTACT, the computer is seen^as just one of several tools, certainly 
not master. It is intended that the information officer should always 
be free to give comple attention, as far as is humanly possible, to 
the caller. - , 
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3.- Speed 

Although it is generally agreed that journals c&n provide an* effective 
current awareness service, this frequently does not meet the practical 
and urgent need for information experienced by a practitioner. 
Information on trends, issues and current events has a certain and J 
justified value but even thi^ can be easily reduced by long drawn 01} t 
publication schedules which can delay the appearance of a research % * 
report for perhaps two years. In addition^ the cost of specialist 
journals is escalating which will, no doubt, eventually minimise the 
practitioner 1 s access to this sort of information; ~ 

Simple telephone based information services remove this crucial delay 
It becomes possible to speak directly to the researcher,- to .immediately 
get the benefit of -expertise and experience. Ultimately the practi- 
tioner may feel tfiat s/he- needs to .refer to written material, but^ 
after having, without delay, discussed the particular problem with an 
expert* 

Such: a system also provides a researcher with some immediate external 
recognition of her work. This is particularly valuable where the- 
development falls outside formally defined research fields - for 
example, the creation of a new teaching or learning technique. 



4. Paperwork 



With this service, there is- no need for 'a postal follow-up or 'computer 
print-out. All the information given ta the enquirer by the CONTACT 
inf onnation officer is recoiled by her a£d is retained so that s/he 
can refer back to it, at any tigie. All s/he needs to remember is the 
CONTACT phone number. Ihis, of course, obviates the need for a 
personal storage and retrieval system for potentially useful information. 
It should also be noted that many academics don 1 t* have secretaries and 
frequently appear incapable of organising and filing material without 
one i 



terrogation ' *f 

Since its inception, one of the most interesting features "of CONTACT' 
has been the role of the information officer, particularly as interro- 
gator and/or counsellor. In, essence the enquirer is helped to clarify- 
hife information needs by discussion with the inf ojgnation officer. 
This brief interaction can include many diverse and delicate elements 
and utilises the sophisticated skills of the information officfer. 
These elevate^ such a simple system into something very worthwhile. 
It could be argued that CONTACT 1 s concern is primarily for the enquirer 
as a person, rather than for "the enquiry. This emphasises the difference 
between this service and, say, \ good library. A librarian is, usually . 
unable to offer the one to one concentration which is certain to occur 
over the phone. Talking to someone within^fhe same institution may 
inhibit the enquirer - s/he may feel nervous of appearing foolish and m / 
thus prefer th<? -distancing effect t>f the CONTACT system. It is " ' X 

impossible for body language to divert "attention and the relative 
anonymity can be relaxing, whilst retaining a highly personalised 
feel. On a very practical level, there may be nO one within the 
9 practitioner 1 s institution able to help in any, way at all. \ 



Such a relaxed exchange can ameliorate any feeling of machine domi- 
nation within this increasingly technological world, and the discussion 
does, of course, lead to a far .more accurate enquiry. Indeed the 
CONTACT officer is ,an interviewer, counsellor, adviser, there to help 
not bully, advise not order and to match the query to an answer in the 
shortest amount tune* i 

Indeed, it is worth considering whether CONTACTS success is related in 
any way to the fact that the information officer is a woman, as is the 
information officer at Imperial College^ CEDAR project?. It could be 
argued that given currents social attitudes, male academics may find it 
easier, to expose their ignorance, and possible weaknesses, to a woman 
who they may, subconciously if not conciously, regard as inferior? 
An interesting thought when looked at in the context of the received 
wisdom that women are "good listeners"! 

Before looking at difficulties encountered by CONTACT, I would like io 
reiterate that CONTACTS role isr * % 

- to promote and e^6ourage the dissemination of experience 
and practical inf ormation; 

- to provide a, fatet person-to-person service 

- to give help wheii^ needed 

- to prevent the generation of even more paperwork 

- to include small localised investigations. 
6« Problems 

CONTACT has, of course, encountered hazards during its development. 
Probably the most potent has been understaff ing which was tlje result of 
insufficient funding. The wgrk is .inevitably labou3>-intensiye as was 
quickly discovered. One information officer and a secretary cannot 
cope with routine office work, inputting and: updating data, promoting 
the service^ scanning journals and periodicals, as well as the funda- 
mental job of answering enquiries. The gratification of providing a 
satisfactory answer is enjoyable but, in this case, has always been 
tempered by the feeling that crucial maintenance work is piling up. 
Ideally, such a project would fully employ two ccmmitteed infojjmation 
officers sharing ideas, plus a well qualified and equally committed 
secretary, s 

Promotion and publicity was recognised as a crucial but complex factor 
for the potential success of. CONTAC.T* Work oh this was, of course, 
constrained by the v insufficient staffing. Since the service is not 
designed to produce an end-product, such as a bibliography or research 
, register, it was decided that a continual but slow-growth approach should 
be adopted. An attempt was made to build CONTACT into the existing and - 
accepted higher education frame work, infiltration rather than hard sell 
and obvious publicity, A two level strategy was felt to be necessary - 
firstly, to make people aware of the service and secondly, to motive 
them to use it* 
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After two years^ it appears the most effective means have been < a well- 
designed information handout; a steady stream of journal articles; 
visiting - to institutions^, meetings, conferences. Use of the service 
certainly seems to be stimulated by personal contact and recommend- / 
ation. However this has been hindered by inadequate staffing ahd . 
funding which precludes leavirfg the office for long periods, as well 
as the payment of travelling expenses. A recorded message is not 
what the enquirer is seeking! 

Despite the attention which had been paid by both Council and CONTACT 
staff to this facet of the work, promotion has not , been as successful , 
as was hoped, even given the limited finances -available. HE insti- 
tutions appear to have an entrenched and suspieious insularity, well • 
demonstrated by an, overall resistance to change^ so CONTACT deliberately 
adopted a -cool, 'rather Ehglish approach as opposed to an all kinging, 
and all 0 dancing assault. Perhaps the economic cvLunate is crucial - . 
a point which I shall be looking at later. 

TWo areas, not directly related to the everyday ^work of the service, 
have proved problematic and are, as yet, unresolved. * These are * 
evaluation and costing. Originally it was suggested that calls should 
be monitored by building in to each working day a "feedback" otime. 
This was soon found to be quite naive. Other *than careful training 
of the information officer so as sAe will-recognise a satisfied 
customer, no workable means of evaluation has been devised. Costing 
has provided an equally thorny problem although the Council is now 
hoping to look at the costing') of its information work overall, and 
this will, of course if possible, include CONTACT. ; 

The use of information technologies is obviously relevant to the design 
of such T a ^service. In this instance, "the new technology 11 is the 
telephone with a computer providing a support service updating, 
storing and retrieving data. It was assumed initially that CONTACT 
users were not" likely to have, in reality, easy access to a database. 
Twenty terminals. in the polytechnic computet room or a terminal in the 
library the other side of the campus is simply not comparable and, 
given the recession, the likelihood of departments, let alone indivi- 
duals, having their own terminals is fast fading. Technical hitches 
al&a create delays. Since CONTACT is supported by a manual retrieval 
system as well as a computerised one, the dhances of i#ceiving an 
'answer quickly are greatly enhanced ara Lare not dep endent on the machine 
being "up". A , 

Conclusion w 

• I . ' ■ 

The establishment of the experimental CONTACT service clearly demon- 
strates some of the obstacles rencountered by any educational innovation, 
and by information services in general in the current economic circum- . 
stances. Despite the increasing recognition being given to the .value- 
of the service, the initial funding period of two years is drawing 
rapidly to an end and it appears likely that *the 'service will only be 
operating three days each week after March. It could be arglied that, 
if the service fiad continued for, say perhaps another 18 months, its 
acceptability and respectability would have become firmly established , 
^ajjd funding more easily found. • < 

This argument is supported by a comparison with the CEDAR information 
officer 1 s future, starting as it did, primarily as a short term solution 
to an immediate problem ie the acute and growing demand for information 
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f * * 

about computer based education^ It now looks as though it will, in ^ 
the future, be well supported by .various sector interests in the * 
education and training world* . Is this because CEDAR 1 s equally- 
effective service is more cpnventional and .thus >more acceptable? 
Is ±t because it had an established base, the CEDAR project, in 
turn, was supported' by the recent NDPCAL programme, on which to 
grow? • 

Attention also has to be given to the issue of charging &r inform- • 
ation, however awkward this is to organise in practice. Does the 
fact that information servipes are frequently free at the point of 
consumption reinforce the belief that information provision is some- 
thing of a^fringe activity? Ironically institutional managements 
are willing to pay for books stacked on library shelves but, short-' 
sightedly, not for such apparently intangible and modern services 
as CONTACT, although it is designed, specif ically to save on these 
very expensive items - staff, -time, paper. The staffing require- 
ments of a telephone-based service are hardly extravagant, two 
information officers and a secretary, particularly if it is agreed 
that they are employed to serve, say, the whole of the higher 
education sector. 

We can only conclude that the harsh economic reality of the -1980§, 
coupled with institional resistance to change,' maRes it virtually 
impossible to create new style information services for education 
and, more importantly, to sustain them and allow them to flourish 
and develop to their-full potential. 
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Reply 

M. C. Head, School of IAbrarianship, RGIT 



Although Jane Hustwit T s paper brief ly surveys the work of the CEDAR 
project at Imperial College, it is primarily devoted to a study of, 
the CET ? s CONTACT scheme, • which is in danger of extinction through 
lack of financial support. This would be unfortunate as the basic 
concept is very soundj accepting that personal exchange is often a 
most successful method of information transfer - a fact borne out by 
Don Ely's study - we see in CONTACT essentially a referral system. 

Thfere are several points in the paper on which I would like to comment: 

1. We are told, that the system is^tructured on keywords but not how 
many caji be assigned to a particular item or whether- there are 

v any limitations. The' term T taxonomy 1 is used but.it is not 

clear whether it refers to a simple classification or a^ structured 
thesaurus, nor is it clear whether the system can utilise and-oiv 

^-^tr^t>gic^ , ^Tr w ite w SB^arches - perhaps" unlikely siiice it is 11 supported 

by a manual system". - — 
• * * 

2. Speed' of response is clearly an important factor in any inform- 
ation retrieval system but, for a telephone service, I would 
question whether the* fact that "most enquiries are dealt with 
within 48 hours" wa? satisfactory. I would consider this to 
be an argument in favour of 01^ line access and feel^that the tf 
paper may under-estimate the growth of availability of terminals. ^ 
Would it not be possible to offer both -an online and a telephone 
service? • 

3. The section on T Interrogation 1 also raises- "doubts in my mindj the 
role of inany libraries is totally misunderstood* and some of Ms. 
Hus twites comments are both offensive and naive j what is here 
presented as a new concept is in fact as old as the telephone and 

* in fact describes the typical reference-' aSid special library situ- 
ation. ' 

4. I would have appreciated some "statistics 'of usej * without some 
context, phrases such as "the number of enquiries rose dramatically" 
are totally meaningless. . -Although one appreciates that staffing 
shortages are due mainly to 'lack of finance, the comparatively 
limited hours of availability >hich results would seem to be a 
further argument for onr-lin^ access. Throughout the paper, the 
advantages of the manual system are stressed but the evidence is 
far from convincing; 'for instance, with only pne information 
officer, one would envisage just as much frustration and delay in 

* making telephone contact as might -be encountered by machine failure 
in an on-lina^ys/tem. 

' ' ' • ' . , '\ 

The conclusions arrived at in the paper are pessimstic, peijiaps justi- 
fiably so given the present economic climate, but*I would still questioi 
the wisdom of the softly-softly approach with regard to publicity and 
promotion; in these times of crisis management, it could be argued* 'that 
it is better td create ft demand which one^knows cannot be satisfied as 
a leveV for obtaining increased finance. " 
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It is also possible that CONTACT has attempted to achieve too brpad *.* 
a coverage in terms of ' subject without first attempting*' to identify 
positive user needs j in her paper, Jane speaks * of "the diversity of 
disciplines and subjects considered as comprising educational tech- 
nology" and this pinpoints the problem. Tfoereas CEDAR has concentrated 
one the one aspect of computer-assisted learning, CONTACT has been 
operating within very vague parameters and may consequently have failed 
to match' some of the guidelines mentioned by Don ELy, particularly 
v those relating to the specific inf ormatplon needs of an identified 
user group. 
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The. Use of Query in Multi-Facet Indexing of Information Materials 

Wo J. Ko Davies, Hertfordshire County* Programmed Learning Centre, 
St* Albans • 



Introduction " , , 

Ihis paper describes an attempt to use an existing computer-search 
programme indexed provide* information about sources of inform- 
ation likely to be of use to teachers at classroom level. At the 
time of initiation it appeared that conventional library indexing 
and retrieval techniques were of marginal relevance to genuine 
resource based learning and also ihat no agreement couldf be reached 
by librarians on a standard thesaurus. 




We therefore suggested that, rath'er than pursue the so fajr fruit- 
less treadmill of an ideal- 1 retrieval-program and a national -da ta^ 
base, we should concentrate on using - and developing - an available 
program_with all^tsjiroblems and advantages. This was acceptable 
because the academic iffEe^bTr^as^^evelop data file structures 
purely for local and possible regional use. Therefore a standard 
thesaurus and nationally compatible file structure were 'not vital. 

Indeed it is hypothesised that individuals or groups can, given a 
working 'structure,- develop their own local data retrieval systems 
with their own local idiosyncrasies. It will be recalled that 
some attempts at this were made - a few years ago but were defeated 
largely tiy'the sheer complexity and laboui>intensiveness *of the,- 
then available, manual retrieval systems such as OCCI and Uniterm. 

For these reasons the availability of the Hertfordshire Query 
programme was taken as the starting po:kit for further' development. ^ 
For those unfamiliar with it, QUERY is a Sequential-search program 
capable of being divided into a number of separate fields fqr each 
record. These fields can be numeric or striiig, can be interrogated 
sidgly v or in multiple and can accept a variety of inputs. Its 
current' liaitajfions include the sequential search naturej- which 
puts a practical limit on the size afid complexity of any given 
datafile; the input technology* using punched cards which deter- 
mines an- absolute top limit of 79 characters per field with a 
maximum number of nine cards per record; and an inability to lay 
out text output except with the use of elaborate procedures,. 

Fortunately our own requirements for -local dataf iles are such that 
only the second restriction is serious and even this is in process 
of being' overcome (we hopet) What is proving^ something of a 
nuisance is the problem of print-out layout. " Computer experts, 
posSibl)r because of .their familiarity with the technology* appear 
to pay r little attention in general to the user yet his need for a ^ 
clerir layout of complex information is considerable. .Conversely 
these who have started with the idea of a clear, layout (most\,6^ 
tKe micro-based programs) find that there are strict limits tiryihk - 
amount of information they can^ include. 
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Progress so far 

Our first attempt was simply to computerise a multi-facet index 
already existing in card form and relating to 626 back-up reading i 
books for adolescents and adults. In retrospect this was rather 
simplistic since each ♦facet 1 was in effect a single term when being 
retrieved (eg, Reading Age; Title) so that the multi-f&cet nature 
of the file was a genuine advantage. Each additional facet interro- 
gated narrowed the area of search. Even the Contents 1 facet, 
arrange^ as a set of keyword topics descriptors were within the same 
field and since it was generated from the books themselves, 

Ihe same did not apply to the next experiment which was to construct 
an analytical data file to retrieve sources of information about 
teaching . In this case we wanted to be able to: 

1. provide a filter system for isolating broad curriculum areas 
(eg,. Chemistry) tfius enabling irrelevant records to be removed 
from review* at an early stage. 

2. provide an analytical device so that an enquirer could, for 
instance, ask for entries on the layoiit of p^int-based work 
materials. 

t 

3. provide a keyword (or cliche!) index to allow queries using 
popular terms (eg. open learning). 

Not unexpectedly this py6ved extremely difficult to achieve, particu- 
larly with regard to 2 where- false drops. could be expected regularly. 
The original idea of combining 2 and 3 in one. set of related fields , 
proved- impracticable and only after 'Several years work has 'what appears 
to.be a satisfactory compromise been achieved, Ihe problem is endemic 
in any* multi-f 3^et base and will, we suspect, never be entirely solved. 

The third structure was really a development of the first and arose 
from a need of our own to index a bank of individualised work materials 
produced by various LEA organisations, , Ihe multi-field nature of 
QUERlT has enabled us here not only *tq categorise the materials by 
content under broad subject and detailed topics, but also to describe 
their intended audience; the uses to which the reviewer thinks the 
materials could be put; the activities (games j information sheets; 
etc,) which the materials include; selected publication data. 
Thus it is fairly easy to find out, for example, what games are avail- 
able on oil exploration suitable for 2nd~y^ar secondary children and ' 
where they can be obtained from. 

The fourth and latest structure is an attempt to provide i^p-to-date 
information' about in-service education activities and sources of help 
available* in a geographical area at a given time. This differs -in 
nature from previous experiments since it has to be a 4ynamic and 
constantly changing file. It algo raises various ethical and * 
political problems. , 

All these experimental files are staall - the* biggest currently holds 
only atycfut 700 recprds - but are enabling us to test out the limits 
of QUERY and, indeed, to spggest certain modifications. They are,* of 
course, only part of. a total retrieval system wfttch may be simply w — 
described as : . ' 
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1. Academic indexing: the process of reviewing and categorising 

the information source. 

2. Technical indexing: formatting the result of (l) so that it can 

be taken into and extracted from a retrieval 
sub-system. 

3. Storing information: so organising the formatted information that 

it is available at any time. to enquirers. 

4. Retrieval: the process of correlating and retrieving 

those parts* of the stored information which 
are directly relevant to a user's needs while, 
.leaving aside irrelevant parts. 

5. Dissemination: the ability to provide the retrieved inform- 
- , . ation in a form acceptable to the user. . 

Currently 1 is virtually independent of any given computer program 
though it poses problems of its own. 2, 3 and 4 are largely determined 
by the actual storage and retrieval mechanisms selected. 5 is done 
largely by paper back-up systems using material selected and photo- 
copied as a result of a computer file search by someone fully familiar 
with the program. Our intention is to make such 'searches easy enough 
to be operated by a direct user (eg. a teacher wanting specific items) 
and comprehensive enough in their results to avoid the user needing to 
seek further information. 

The problems we face 

In comparison with other f local 1 programs we have seen; QUERY provides 
an extremely flexible base provided the user is prepared to take a 
little trouble but its strength is also its main Weakness - its very 
flexibility means that, if one yses its potential to the full, there 
are user problems. In particular it is very difficult to get a tidy 
" display layout unless coding is used extensively - and that in itself 
is not helpful to a user. Thus in BETH, the index of sources on 
teaching, one cannot practicably put in DESIGN of INSTRUCTIONAL 
MATERIALS but must say DE:IN:MA. The print out, because of the mass 
of data available, is very likely to wrap-around especially on a VDU. 
By its nature the program is aWo very dependent on user awareness. 
Unlike Dialogue - or* for example the EL project - it is not structured 
t to ask or prompt the enquirer, nor will it automatically do analytical 
searches in related fields (for example the BL experimental program 
will isolate a k^y term in bath title and topic ^fields 1 aiid inform 
the usef of the frequencj^in both while effectively saving the records 
in a* sub-file. To use QUERY in a siMlar mode one would need to 
interrogate both fields - QUERY TITLE sub 11 TEACH 11 or PROCESS sub V 
. Tt TE" - and then tell the computer 'What details of the ! hits ! you want 
printed 6ut). Further more input technology using punched cards 
restricts the number of characters ;Ln aiiy given field •*^~Thus at 
present the system requires considerable paper back-up in the form 
of uod&~ lists; subject and topic lists; and abstracts. We 'are 
designing computerised equivalents - a datafile of abstracts which 
Jean be from any« data' base and a data file which Will enable users to 
/ interrogate the current topics lists of any given file - but these in 
J themselves require extfra work on the part of abuser. - We are, there- 
^°^ e * n0t ,at ° VCr destination. 
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Even so we have got far enough down the road tp be able to stee where 
we ape going. The nature and complexity of QUERY, originally 
designed for use on a powerful^mainframe, makes it slow to operate on 
current micros. 'It would seem likely that, in the near future at 
least, the best plan will be for a user to interrogate the main file 
either 'or*- line 1 or 'by batch to mainframe and then put up a selected 
sub-file for local use. An alternative mode of operation is for 
users to go through a referral point which will carry out searches 
for them and provide the results in paper ^orm, but .this does lose 
the immediacy of response which we find So useful. 




♦ 

Reply 

R 0 L 0 McMorran, Aberdeen College of Education Library 

Mr. Davies has suggested that local or regional data retrieval systems 
should be developed, rather than effort being, wasted on fruitless 
.attempts to initiate national databases which cpuld satisfy the inform- 
ation needs of alJL users. Two o£ the advantages of local s$tems 
have been mentioned - first the availability of existing programs 
(and computers to run *them on) and secondly the ability of local 
systems to develop 'local idiosyncrasies 1 , and if by this is meant 
the indexing of local materials, such as teaching materials on area - v 
studies, or say, collection of reports on localised educational 
problems, then I would agree that it is a considerable advantage. 
Other advantages which I might mention are the lower costs nowadays 
of computers which could be used for infortnaton retrieval, and the 
fact that in a locally-based system it might be easier to identify 
the needs, and the changing needs, of specific group? of users. - ^ *? 

Nevertheless, I have some reservations about a multiplicity of local 
systems being established in isolation from each other, perhaps 
duplicating the work done, each using different programs, with 
differing inputs, differing search strategies and differing functions. 
There is neither the time nor (I am sure) the necessity, for 

re-iterating the merits of centralised information retrieval units, 4 

in theory, for the detailed subject indexing of large numbers of 

items to be used for educational research, which could be available 

on-line to regional centres. Perhaps a case could be made out for 

a national information retrieval system, such as that described in 

the Scottish Council for Research in Education report 1 Information 

retrieval in the field of education', published in 1974* "which could 

be complemented by local centres which could provide access to national^ 

as well as to locally-produced files. * 

I think I should at least mention here the availability of ERIC. 
In Aberdeen College of Education we have found this usefid on - v 

occasion, and both Jordanhill and Notre Dame Colleges of Education 
have decided that it is worthwhile to initiate the installation of 
computer terminals which would probably be used mainly for access to 
ERICy' Our enquiries are howeVer not specifically in the field of 
educational technology, and our research users are not primarily 
1 teachers at classroom level 1 • 

Hie needs of users are rightly stressed by Mr. Davies. Desirable 

qualities mentioned are a clear easily-understood output, immediacy 

of response, availability of materials indexed, sufficiently large 

files to satisfy enquiries, and direct operation by tfie information 

user. From the description given in the paper and after reading the 

user's guide to the QUERY package, I would suspect that there is some 

way to go before this last quality is achieved, nor am I convinced 

that it is a serious handicap to go through an intermediary, providing * 

the information inquiry is plearly formulated and the intermediary 

readily accessible. Perhaps it might be worth while itemising one 

or two other user requirements. References produced should be relevant 

to the inquiry. . The materials retrieved* §hould be of a high quality. n 
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Precise enquiries .should be capable of producing precise responses* x 
I feel that QUERY may have some drawbacks in these areas • The 
thing which surprised me most on, reading Mr. Davies f paper was his 
dismissive remarks on aspects otf information retrieval which are 
generally reg^fded as being of central importance, specifically the 
use of carefully constructed thesauri and 'the use of computer search . 
strategies in information retrieval (I take it that this is' what is 
meant by saying that conventional library indexing and retrieval 
techniques were of marginal ^relevance). Standard thesauri, such as 
the ERIC list of descriptors or the EUDISED thesaurus may not be ideal, 
but their use at least saves one the trouble of constructing a 
thesaurus for oneself* Without a thesaurus, problems will inevitably 
arise, from synonyms, words with related and perhaps overlapping 
meaning, broader arid narrower terms, and terms with vague or ambiguous 
meaning (not uncommon in education). The more precise the indexing 
and the larger the file, the more essential is a thesaurus.*. Finally, 
I would like to tentatively suggest that an interactive system might 
be of more use to users, and that a system using file inversion rather 
than sequential search might give more, scope for specific information 
retrieval. 




\ 
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CHAPTER 5 



4 . THE EXETER ABSTRACT REFERENCE SYSTEM ON HIGHER EDUCATION 

' — ~P — - ~ \ 

. D. Bligh, Exeter University Teaching Services 

The Exeter Abstract Reference System (TEARS) is a computerised retrieval 
system developed .by Denise Cannon of Exeter University Computer Unit to 
serve departments within the University. It is adaptable and can 
contain files storing different kinds of information such as records 
of films or bibliographical references. This article describes TEARS 



on Higher Education which primarily stores abstracts of research into 
teaching and learning into Higher Education. 



*esy 
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I was appointed to provide services to University teachers in various 

ways. These included the provision of information and working with 

\ I 
colleagues, particularly new staff, to develop university teaching 

techniques and innovations. ^ 

In 1975 there wen^certain local conditions which made the establishment 
4>f a computerised retrieval system seriously worth considering. The 
Computer Unit was interested and supportive because^ it wanted to provide 
a general facility for all departments. Also at that time I was probably 
better acquainted with the research literature on university teaching 
than most people. There was no British retrieval system in this field 
and, as I shall describe, it seemed that there could be good reasons to 
have one, but only ofl|\ Finally there had recently been growing 
acceptance that research into N teaching and learning in Higher Education 
is both interesting and relevant to practising teachers. 
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I. WHY HAVE "TEARS"? . 

However, while these factors made the establishment of a retrieval 

i » * * 

system seriously worth considering, ^they were notiin themselves 
sufficient to make me embark upon such -a long term commitment. There 
were general policy issues concerning (A) the academic acceptability 
of teaching development, (B) establishing a research orientation to 
teaching services, (C) the development of teaching] services as a fielcf 
in itself, and (D) the possible services to which a retrieval system' 
could contribute. 



I shall now consider each of these 'in turn. 



ESTABLISHING THE ACCEPTANCE OF ACADEMIC SERVICES 
1. Research a common value . . 

It is well known, that staff development is an unpopular cause within 
universities, partjcalarly a fairly conservative one like Ex&ter. It 
seemed important to build, upon values' academio^ r taff~ would have' in 
common. Research is one such' common value, It would be hypocritical 
for an engineer, chemist or lawyer to vtflue hisTovyn Research on the 
grounds of 'its professional utility and\to,deny in principle the 
relevance of educational research io his own profession./ Similarly, if 

a historian or linguist values the v accumulatidn cjff knowlege and wisdom 

\ * • ? * 

in his own field, it would be hypocritical to« deny in principle the same 

"possibility for other fields. Thus whether, research is' valued* for its • 



own jake or for its useful ness* the same value can be*applied to 
educational research. * / 



2. Know! edge. appears* neutral - no threat 

Compared with many activities of those concerned to improve teaching in 
Higher Education, the accumulation of research information appeared to 
offer little threat to/ academics' self-esteem. " Workshops on lecturing, 



for example, may have a- "holier than thou 11 appearance. Furthermore, 
when one is hew to an institution it is very easy to tread on other 
people's unseen corns; but the collection of information seems less 
likely to do so. It is not prescriptive; .it provides support for 1 
academics freely taking their own decisions. 

3. Reading acceptable to non-attenders 

It is common experience amongst those concerned to develop university 
teaching that less than 20% of ^academic staff will ever attend any 
meeting or activity one cares to organise. The fear of exposure does 
not apply to reading, provided no great, effort is required/ For this 
reason reading should be in small quantities. It s&emed that abstracts 
of research, unlike extracts from books, could be concentrated and 
sufficiently suggestive to provoke discussion. 

-* 

4. Slow cognitive dissonance * 
The problems of acceptability and the development of teaching are 
problems of attitudes and attitude change. Attitudes^are said to 
change by a process of cognitive dissonance. They change by a process 
of continual adjustment toreconcile "dissonant" information, I had 
no dogma to which I wished to convert my colleagues^ Still less^did I 
expect sudden conversions. I had a simple faith that if one could create 
ah academic -community well informed on educational research', academic* m'I 
decisions wduld be better informed and wiser. 

"V 

4 

5. The unit is to the university as the university is to society 
The role of a teaching services unit within a university is like the 

role of a' university in society. In holding this simple faith I ascribed 
to nothing more than the beliefs of universities themselves. One of the 
^functions of universities is to collect and disseminate the-products of 
scholarship. Just as universities inform the community about the world' 
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in which we live, it is part of the function of a teaching services unit 
to inform the academic community about the study of Higher Education. 

* *■ * » * 

*~ 

(B) JO ESTABLISH A RESEARCH' ORIENTATION 

A further set ofi^ reasons for establishing a computerised retrieval 
system of abstracts on research into teaching and learning in Higher 
Education is underlain by a policy decision which is very evident from 
what I have already said: the Teaching Services Centre should have an 
academic base if not a research orientation. " Two of these underlying 
reasons may be made summarrily explicit. 

1 . To inform academic decisions * 

There needs to be a standard way of retrieving information that will 
inform the way teaching and administration a^e carried out. * ^\ 

* • • ' 

2. Universities' as objects of enquiry \ 

* Universities need to understand themselves as well as the rest of the 
world. 

i 

3. Base services ors research, if possible 

Similarly, if hypocrisy is to be avoided, teaching services should be 
research informed where possible. In the same way that universities and 
society itself are constantly changing, the Teaching Services Centre 
was young, evolving rapidly and needing to do so in\^n informed way. i 
do not mean by this that we needed to be able to quote chapter and verse 
of research abstracts for everything we dip. That was obviously 
impossible. I mean that, just as the study of literature or sociology 
may give an understanding of why people behave as they do, so scholarship 
in the field of Higher Education provides a climate for wisdom and 
judgment c fn the provision of teaching services. 

v ■ * 
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Naturally when members of the Centre work in consultation with academic 
staff they want to support their. ideas with research evidence. TEARS 
can be used to provide .this support. 

y » 

4. Widespread literature • # 

the literature on Higher Education is fTGng exceptionally widely in 
remote professional journals as well as educational ones. This means 
not only that academic staff would n^MMnd relevant literature even if 
they had the, inclination to look for it; but that if members of 
Teaching Services are to provide an information service they will need 
someway of selecting what is relevant. This is ,the work of a retrieval 
system. * 



5. Our ignorance 

Furthermore, no-one working in the field of Higher Education c^n know all 
the research relevant to every educational service that may need to be 
provided. Some way of retrieving what is relevant, and only what is 
relevant, is required. * * 

ESTABLISHING THE FIELD - . . 

I was conscious in my new job that I was a member of a new profession. 
I saw. myself not so much as a "staff developer", as a "teaching developer 

* - 

- although there may be some overlap between the two. Call it what you 
will, if there was an emerging profession, there were three problemsjji 
each of -which an informa-tion retrieval system coul^Uplay a part. 
1. Delineating the field 

Presumably the*area of competence of this profession would be delineated 
by the limits of its members 1 knowledge or expertise. This of 
course to say that all the knowledge would be unique to members of the 
profession any more: than medical knowledge is only possessed- by doctors, 
[jevertheless, the earmarking of certain knowledge by putting it in the - 



retjieva? system would label it as relevant. I am not of ''cpurse 
suggesting that the professional expertise of teaching cleveTopers consists 
solely of knowledge that can.be put in a retrieval system any more than 
the skill of a doctor can be written in a textbook. On the contrary, the 
skill of a teaching developer is much of what he has to offer. Nonetheless, 
the knowledge base of the profession helps to define that profession and 
de-limit its areas of competence. 

1. The problem of authority ' - . ' \ 

The knowledge base of a profession is a source of its •authority. For 
~mostj professions this i s established~slpwly and they go through early * 
stages during which their knowledge is challenged until there becomes 

an established professional body which is thd respected arbiter of standard 

<* - • 

Bodies such as, the various institutes of engineering, the Law Society and 



the British Medical Association constantly strive to maffStain and raise 
the professional standards of. their members by monitoring the knowledge of 
entrants to their professions. Teaching developers have no such 
professional body. Consequently the consensus of research findings may 
for a time be a support to their authority. the use of a common retrieval 
system by teaching developers _ could- be a way of accelerating professionaV 
consensus.' This would not only fuHtter the^ professional identity of , 
teaching developers, But 'raise their academic standards by freeing them, to 



explore deeper or wider- issues.. 
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3. Discipline - paradigms and picture building r */ 
Most important of all I saw the use of retrieval* systems, as essential to 
the development of new paradigms for educational enquiry. This is a big ^ 
issue and really requires a separate paper, if not a separate book. 
Nevertheless I will attempt an outline sketch of my view. 
> 

In the use df educational research tfrere are problems of generalisability, 



conflicting results and simple' lack "of evidence. I must .ask you^to 



consider the field of education as a picture consisting entirely of dots. 

The dots are experimental results and all kinds of other information. 

Where we have no information there are no dots on the picture. In other 

places the picture is crowded with dots of differing completions, The 

• « 

colours vary in their intensity with their credibility. 

"The picture requires interpretation. No one dot, nor even a few dots, 
'are sufficient to judge the outline of a shape on, the picture. Most of 

the picture is blank, but J f we step back, some general shapes can be 

discerned. 



The construction .of the picture is never finished. Although a more or 
less scientific approach may be desirable to specify the dots, their 
interpretation is an art. 

Next I must ask you to think of teaching as a decision making activity. 
' It is one of the management professions - the management of learning. 
When-taking a decision we need only look at that part of the picture 
which is relevant. The decision to be taken - the question to be asked 
prescribes the relevance of evidence. ^ 

' Wha't has all this got to do with retrieval systems? The information 

store "is the picture. The informatifcn looked at- depends upon the auestion 

* ■*"»(.) ■> > 

asked. One way to interpret the information's to tabulate it in columns 

\ * / * 

^_ ' according to. the answer or decision itQmyiies. 

(D) POSSIBLE SERVICES ••" . • f ' 

Apart from theoretical issues I 'thought'there were very practical reasons 
for having a computerised information service. , ' 



d 1. Individual needs/interests vary ' 
£^ Retrieval- systems can produce individualised summaries of information 



<while conventional printed'material ei ther'forces academics to take a 
diet only some of which they may want, ar to reject it altogether. 

2. Interest bank . / 

By keeping a regular search\prbfile for different individuals it is' 
possible to see who has common interests. In this way teachers wi^h 
similar interests across the campus can be put in touch with each other 
or meetings can be arranged with them specifically in mind. 



3. "Contact" , ' f - 

The national telephone information servifce, "Contact", funded by the ^ 

Council for Educational Technology and the subject of another paper at 

this Conference, is partly a development of the interest bank idea. 

» 

Even without this person to person information service; given su 
resources, TEARS could be used nationally. 

* V 

4. On demand * 
One advantage of the retrieval system is that it can be used dn demand, 
while there is always a risk with printed literature* that academics feel 
it is being rammed down their throats and get in the*hab4t of losing the 
waste paper "basket. 
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In principle the 'oh demand 1 facility cduld include onrline provision of 
„ information to seminars or other activities organised by the Teajchyig 
- Services Centre. , * ■ 

5. Multiple classification and multiple catalogues 
It might be said that an information system on card^ would be ju^t as 
good. While it is true to reply that the abstracts vary greatly! in 
length so that cards are in appropriate, this reply is trivial. whe 
• attraction of the retrieval ^system lies in the possibilities of multiple 
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classification and multiple catalogues. Any given abstract may be 
assigned an almost unlimited number of keywords by which it may be accessed 
while each reference in a card index does not normally appear* in more than 

0 

two or three places. Furthermore the retrieval system can contain more 
than one catalogue. For example we may only wish to consider research 
findings, sometimes 1 * only reports of innovations in teaching and sometimes 
both." It is a simple matter in a computer search to exclude all of one 
category or another. , > . 



\ 
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II. WHAT 00 'TEARS' RECORDS CONTAIN? 

The answers to tTiis question, are «et out summarily in Figure 1. 

Authors -are normally given with their surname following by their 

initials. C. R.A.N, stands for Computer Record Acquisition Number,, 

The title is normally the title of the research article but in some 

t 

cases we, have abstracted chapters of books separately in which case the 
title of the book occurs in the citation. 

A coden is a shortened code used by librarians to indicate specific 
journals. The use of the coden is simply to save computer space. 
Furthermore some journal titles are rather long and similar, and some 
are identical . 

* * 
• The acknowledgement gives the initials, of the a^Eractor or the source 
\^Jf the abstract. x . 

The keywords are 0 selected by the .abstractor. . There is considerable 

stetTI-tfi doing this. Unlike most retrieval systems we did not set out 

with any theoretical .notions about hierarchies- of key words or their . 

logical relationships. . But we have'' recently been through a pruning 

process. The computer- itself is capable of producing hierarchies and 

• : • of detecttfig' errors. For example* it can tell us how-many tfmes the 

key words 'Lecture' and 'Teaching Methods' occur together and how often 

/each occurs separately.' Where .the word 'Teaching Methods' does not 

occur with the key. word '.Lecture' this can be ^corrected. 

r . / ' 

y - y.% 



V 



The abstract is normally either written under the single heading, 
'Synopsis', or under the .experimentaT headings listed-j-n Figure 1. 
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Figure 1 »- v What TEARS. Records Contain 



Author 1 


/ CRAN 

> 

* 


Title 


Journal or Coden, 




Place where research done • 




Department-. 




Institution 




Country 


- 


Year(s) of publication 




. *Citati on (volume, issue, pages, 


other locations) 


Acknowledgement v 


* 


Keywords 

Abstract: Synopsis or Aim 


» 


Sample 


• 

* * 


Method 




t . 

Findings 




. ' Comments 




s 

* not searchable 
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III. SEARCHING 'TEARS' 

The information base is searched by the use of key words. These are 
normally joined by logical operators 'AND' , 'OR'' and 'NOT'. Thus if 
you use- the key words LECTURE^'AND' DISCUSSION the retrieval system 
would give you all research reports containing /both these key words. 

D 

If you searched using the profile LECTURE 'OR' DISCUSSION it will give 
you all records containing either or both of these two key words. The 
use of 'NOT' is* to 'exclude records with particular key words. 

In addition to using the key words it.is also possible to search using 
part of the bibliographical record, except .the citation and the abstract 
itself. For example, it 4 is possible to search for, or to" exclude all 
articles by a given author or combination of authors,' to exclude articles 
from a particular country such as the U.S.A., or a particular type of . 
department (e.g. Psychology Department): Similarly it is, possible to 
searcVfor, or exclude, articles published before or after, or. between 
certain years. Similarly abstracts from particular sources, such as the 
Society for<»Research into Higher^Education abstracts, can be specified. ' 
Again it.is possible to consider only records before, after or between 

particular CRANs. $ ' 

>* • 

For all these things there are dictionaries indicating the number of'times 
.a key word, author, etc. occur within t£e total information store. 

- A 

The information base may alsb be searched using fragments of key words. 
For example if you were interested' in 'lectures', 'lecturing' and 
•lecturers', it would be possible to, search for them all using 'LECT*'.* 
This will produce any record with a key word beginning 'lect'. Jt would' 
not, of course produce records using key words such as 'taped-lecture' 
or 'new-lecturer'. To include these the asterisk will also need to be 
placed at the beginning of '*LECT*'- . However this would be a mistake. • 



The computer woulcNHso give you every reference including, the word 
seLECTion!! * . .• • 

To overcome this mistake there is also a fragment dictionary in which it* 
is possible tp check for errors of the 'SELECTION 1 type (see Figure 3). , 
This shows th&t the fragment ^LECTUR*' would avoid all references to 
do with 'selection 1 being printed. 4 

In the first search profile written in Figure'2 the searcher was 
interested in comparisons of the effectiveness of Lectures and discussions. 

» o 

of 

Figure 2. Three Search Profiles 

— ~ ; ■ * * 

COMPAR* 'AND f LECTUR* 'AND' DISCUSS**.. 'AND' EFFECf* 
(DROPOUT 'OR' WASTAGE) 'AND' UNIVERSIT* 'NOV 

(POSTGRADUATE "'OR' DOCTORAL) 
(*EXAM* 'OR' *ASSESS* 1 IGNORE' TEACHER-ASSESSMENT) 

'AND' (*ANXI/ 'OR' STRESS) 

y 
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VIDEOTAPED- 



N£W- 



MACRCMO- 
APPLIC AT ION-OF-KNOW 
CURRENT- AFFAIRS- KNOW 
KNOW 
PREKNOW 
SUBJECT-KNOW 
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* KNOW 
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. ' SOCIO- 
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. COL 
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LECTURE 
LECTURE 

LECTURE- ATTENDANCE 

LECTURE-COURSE 
LECTURE-DEMONSTRATIO 
LECTURE-DISCUSSION 
LECTURE-MATERIAL 
LECTURE-METHODS 
LECTURE-ROOM 
LECTURE-SCRIPT • "* 
LECTURE-STRUCTURE "* 
LECTURE-STYLE " ' 
LECTURE-TECHNIQUE 
LECTURE-TIME 
LECTURE-WORKSHOP 
LECTURER 
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LECTURER-TEACHER . 
LECTURER-TRAINING 
LECTURING 
LECULAR-SGIENCE 
LEDGE 
LEDGE 
LEDGE 
LEDGE 
LEDGE ' 
LEDGE 
LEDGE 

LEDGE-ACQUISITION rf 
LEDGE- ADVANCE 
LEDGE-APPLICATION 
LEEDS 
LEGAL 
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LEGAL-STUDIES 
LEGAL-TEXT 
' LEGE _ 
LEGE 
LEGE 
LEGE / 
LEGE ( 

LEGE ( « 

LEGE. 
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LEGE 

LEGE ' ' 
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'LEGE-AGE 
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INVO 
SUPP 
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COMP 
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COURSE- IMP 
PERSONAL-PROB 
PROB 

PSYCHIATRIC-PROB 
ACADEMIC-EXCEL 
EXCEL 
VIO 

SCRIPT- 
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INFU 
TE 
^TE 
TE 

INTO 
TO 
TO 

, ACCE 
ADO 
HELP 
ATH 
COMP. 
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PAR ' 

NEWS 
COURSE-RE 
RE 
RE 
A- 

ACADEMIC-— 

ACHIEVEMENT- 
ACTIVITY- 
COURSE- 
CREATIVITY- 

EDUCATION- 
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LEGE-OF-FURTHER-EDUC 

LEGE-OF-HIGHER-EDUC A 

LEGE- PL AN 

LEGE- PREFERENCE 

LE9E-SCHOLARSHIP-SER 

LE6.E-SIZE 

LEGE-TYPE 

LEGES 
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LEGES-OF-EDUCATION 
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LETE 
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LEVEL 
LEVEL 
LEVEL 
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Figure 3 : An extract from a fragment dictionary 



IV. QUESTIONS PEOPLE ASK ABOUT 1 TEARS 1 



The questions people ask about TEARS are bipTadly of two kinds . There 

are those about«decisions we had to take to construct and maintaiff'the 

system. There are other questions whi'ch are evaluative. The two 

kinds overlap. * - 



(A) QUESTIONS ABOUT DECISIONS TAKEN 
1 . What to .include , s 
The most comfflon .question is how we decide wihat'to include in the system. 
Do we filter material on the basis of its quality? If sfb 9 how are such 
judgments made? Do we include non-research? How much detail do we 
include? What are the boundaries of the field "higher education'*?. 
Does the system have concentrations in specific areas? 

Inevitably there has to b^some judgment to exclude bubli cations of 
indifferent quality, but there can be no single criterion for this. 
Educational research includes many different paradigms^ On tjhe whole 
the quality control is exercised- when deciding what to include in the . 
abstract, rather than the decision to include the reference at all. 
If we can find nothing worthwhile to include in the abstract then 'the 
* article itself is not included. This means that we cast our net quite 
widely (there is alotof educational research which includes one or two 
worthwhile ideas within pages of boring rubbish) but at least the 
bibliographical information gives the user the opportunity to chase up 
the references for himself. . 



At first we did not include non-research items, but with the publication 

of the Nuffield Newsletters on Innovation in Higher Education, we included 

details of innovations and v w£ have always wanted to include scholarly 
d intuitive work of a non-empirical kind. . 
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The question of the amount of detail to ificlude has. always been difficult 



: and the answer depends upon the skill of the abstracter. I have always 
— beeiKutterly irifuriated by the ERIC abstracts which either. tell you 
nothing at all, or that the article deals with a toqic without giving - 
any indication what it^says. One of the reasons for establishing our 
own systen/and not relying on ERIC was precisely that I wanted a system - 
to incWfe as much information in as little space as possible. TEARS 
can include diagrams - anything a computer can print - but they tend to 
be expensive with computer spaced Statistical information is very 
economical and so we have not been afraid to include^ significance levjiljsf^ 
etp. in our abstracts even though numbers may Wghten^ome of our arts 
Colleagues. Nonetheless, statistical surveys present a pifoblepi one 
- /cannot reproduce their whole data base. One has to be selective by 
giving totals or mean scores; but, for example., it is oftterv preci^€T^ the 
specific correlation between f A' level and Degree perform&xce 3n a user's 



particular subject that the user wants to find out 



i 



^ There is no doubt that the system started with heavy concentrations in 
my particular field of interest, teaching methods. More recently, this 
imbalance -has been reduced and we have often searched the literature in 
response to enquiries and only subsequently added the abstracts to TEARS. 
Indeed, it is arguable on .groitads of cqst, that this is the way we should 
* proceed: store information that people have wanted to know rather-^fehan 
continuously carry out a massive operation in the hopes of covering 
everything. This is now our policy; but jt has the two-fold disadvantage: 
first that the response to users asking new questions is not as rapid'as 
they expect from a computer precisely because it is not a computer 
responding; and secondly TEARS itself gets a bad name because the user 
learns that it contained little or nothing on his subject. Nonetheless 
we do our best to give him, the information. 



9 
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2. What key words? 

We did not, as many either systems have done, take detailed decisions on 
our system of key words before establishing the data base. To many 
information scientists this wou4d be sheer lunacy; but the TEARS system 
is so adaptable that any abst/act can have approaching 80 key words and 
initiaTly we adopted the policy of "if in doubt, include it". This 
meant that the number of key words in use grew very- rapidly at first, 
but then b$gan to^Veyel off to a gently rising plateau. At this point 
we embarked upon the task an information scientist normally tries to 
, avoid - going through all records and rationalising the key words. This 
meant eliminating many keywords by substituting alternatives already in 
use, establishing cross references for these synonyms and including, 
where ;they had been omitted, those general terms, like "Teaching Methods", 
which are higher order key words in hierarchies of concepts. This, of 
course, is to ensure that someone using the key word "Teaching Methods" 
does not also have to specify all the different methods separately. 

* 4 You will see from this that our approach to key words has been empirical 

A 

rather than theoretical, axiomatic and prescriptive. ^ 

3. What are our sources of information? 

k When TEARS was first established I possessed a large number of "abstracts 
on methods of teaching, assessment, student selection, independent study 
and evaluation together with some on student types. In virtually all 
cases I had read the original work myself and could vouchsafe for the 
quality of the abstracts. ^ 

Obviously it was impossible to maintain this workload and we soon reached 
an understanding with the Society for Research into Higher Education that 
we could include their abstracts, possibly modified to conform to t>ur 
ERXC^ 0rn,at ° f headings * ^ n c° nc fi tior \ that we paid them a royalty on any income 

BR. 



we might derive. The Centre also receives copies of the abstracts made 
at Kingston Polytechnic together with the College Student Personnel 
Abstracts which are very good, but seem to be little known in this country. 

4/ How is information put in? N 
Nowadays we, simply type the information directly on to an archive file 
by using a computer terminal- * Blocks of up to 20 records at a time are 
then checked before being added to the main file. This reduces the 
likelihood of any mishaps with the main fite, 1 Experience shows that 
the rules for inserting new information can be 'learned very quickly, 

5, Do you produce summary publications? 

So far we have not produced books of abstracts on particular topics, but 
it jis something I have always had in mind. Once again vt is a matter of 
deciding one's priorities, 

EVALUATIVE QUESTIONS ' * 

1, What cost?. What charges? 4< 

Theoretically -the University should charge the going industrial rate 

for any computer services to users outside the University. In practice 

these charges" have never been applied to persons from educational ( . - 

institutions. 

Bearing in mind that TEARS is a general system developed by the Exeter* 
'University Computer Unit for the benefit of all departments , at the 
University, the development costs .for TEARS on Higher Education, are quite 
Tow,- So far^we-have used staff paid .at a secretarial _raite, although, 
those who have developed the system of keywords are clearly no ordinary 
secretaries! Obviously the ctfief cost is labour, and when the system. 

4 

first began I aimed for three or four records to the pound. Presumably , 

* ' \ . t , * ' * . • . 

inflation has now played havoc with thi,s. 



2/ What is i ts rate of growth? 

! — T t ~' 

It was always my hope that the system would grow at up to- -10 records 
per day or 2000 per year. This soon proved absurdly ambitious because 
it totally ignored all those extra jobs that inevitably so with an 
undertaking of this kind. -It assumed that time devoted to the system" 
was used for nothing but written abstracts and typing them at {he 
terminal . 
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For the past couple of years the system has beep_in abeyance, and 
virtually no abstracts have been added. S"he\centre simply devejope'd 
other priorities. In particular^ we were invited to develop the 
"Contact" system-which is more interactive, individual and, above all, 
more human, 

3. Who are t he usepsi 

- / i * 

We obtained a list of members .of the University who use the system x 
regularly, together with a record of their interests. However we 
obtained- this Tist'By circulating a questionnaire and this was a mistake, . 
'because in many cases the keyword formulae they submitted, contained 
errors. We have not encouraged a great many users from outside the l 
University and consequently the system is in relative disuse. 

< - * 0* 

4. ^ What is your error rate? 

% * / 

The error rate clearly depends 'upon the suitability of the keywords 

. •* . 

associated with each record and tha suitability of ttje search profile 

that is used. 1 One^ngineer who wanted to test. the system by searching 

'in an. area with -which he was very familiar told us that 27 out of 28 

articles printed were on targets In "contrast another lecturer^ whq^as 

interested in language laboratories, aske^ for everying on !]aboratoriefe ' 

and everything on 'languages''; the result was-an~en"ormous irrelevant 

printout. Originally I thought that if a printout contained one in three 



records relevant to the user's area of interest, the system would be 

» 

acceptable. In practice^ provided the,^earch profile is wisely thosen, 
the" success rate within the University has been better than this. The 
same. cannot be said for outside users, 

« 

5. Do* you offer a .user consultation service? 

The reason for this discrepancy lies in the user consultation that is 
possible within tffe University. The language of Higher Education^ is 
not the specialist language, ofv the ordinary academic. Consequently he 
cannot always choose his search profile wisely. It might be. thought^ 
that the need for user consultation reduce$/fetra attractiveness of the 
system. the corvfcihary is the case. Personal contact is crucial to 
*user satisfactiort. Consul tation^of ten gives the user insights which ^ 
no infgrmbttdn service alone could possibly provide. * 

* 6. What fs yout^aluation of TEARS? 

My -evaluation wiTI depend upon the criteria I use. Using some criteria 
one must say ^hat it i's a failure. It is* doubtful whether the value 'of . 
the information supplied exceeds the cost of establishing the system; . ^ 

nng vtv. it 



There could probably have been cheaper ways of supplying — =irrone 
sense the fact that we are no longer able to update the system is a , 
confession of failure. On*the other hand, the system is stiM usable / 
and informative.* ' . * 

But it emphatically does not follow from this that the enterprise was 
not worth undertaking or that the decision to establish the system y/as 
wrong. * There are many decisions one takes in life in which one knows 
±he chances are against success but which are -nonetheless Worth taking* 
either because the benefits could be enormous, or because one sees t%y 
are a move in the night direction*. Many flying machines were designed 
;and failed before man learned.to-fly; but I cannot say that these 

~" £R. 



attempts should never have been made because they were likely to.be 
unsuccessful. To have failures of this kind is a normal part of 'the 
process of education - as anyone who has taught 3 'five-year old 

arithmetic or reading will know. It is experience that will be built 

i . ■ > 
upon because it is experience along the right lines. I cannot doubt 

'that educational decision making will one day be heavily dependent upon 

information retrieval systems. To give up at the start is no way to 

accomplish this, vision. 



There ha^e already been spinoff benefits and there are more to come. 
The "Contact" information service grew out of this experience. The 
development of TEARS gave Jessica tlaridge knowledge and experience on 
which she built tjie "Contact" .system. Parts of the TEARS programmes 
have been- adapted as an indexing service for academics writing their 
own books r ( 

If TEARS makes a contribution to the development of viable paradigms 

J 

of educational decisionmaking, its value will vastly exceed any • 
expenditure upor/ it. . * & 



My point is an aid one: you cannot measure the value of research and 
development either in isolation, or purely in terms of their immediate 
consequences. Human knowledge and human endeavour are cumulative and 
interactive. 



I hope it is clear from this paper that the creation of JEARS was forward 

looking. It was~not so much my purpose to .collect information about 

research in the 'past, as to establish a professional base in teaching 

development, contribute to new standards in educational decision making 

and take a'step towards the development of new* paradjgrfis in educational . 

' - / ' - * / : ' . ***** " 

enquiry. Thus its implications go far beyqnd the provision; of /information. 
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M. ROEBUCK, S COTTISH EDUCATION DEPARTMENT 

Two introductory remarks: the first about the way we are discussing 
the topic. The context is information retrieval in educational 
technology. Can I ta*e this to be onl^ a context and equivalent to' 
retrieval in other areas of education? Some confusion seemed to be 
arising in the first discussion between the role of the librarion, the 
role of an educational technologist, and be role of a user. The last 
two are the same in my understanding. * • 

Second point: my surprise at the comments just made about the 
difficulty of s|rttingj^inter e st among university staff in teaching 
methods ^nd the apparent-new discovery about the sensitivity of staff 
to information along these lines. Teaching in higher -education was low 
status. Teaching in higher education can be regarded as .being a rather' 
muridane thing within the immediate domain of the HE teacher and it is 
*ndt surprising- that Donald Bligh could not find staff who" were willing 
to share their problems. I am always intrigued by the ability of HE 
staff to ignore the existence of relevant papers and research. Don Ely- 
results support this. The work done by Clarke and Dunn/Gilbert also 
support and I recall a conference on programmed learning in chemistry in 
universities in Scotland in 1965 which was based entirely upon.one-TES 
article without any taking up of any references. 

* 

The paper by Bligh reviews ..... « n d gives a useful listing of some of the 
problems in this area of research orientation. ' 

^"responding I found the paper difficult to apprdach: is it a self- 
confessed failure? v " 

Or does it have lessons for us? 



Or are those tap* TEARS 1tself Sh ° Uld T*" 
by readinsthe literature, or making CONTACT? I do not believe; 

things weri so different 5 years ago. 



♦ 



« standpoint inithis context is M a user 'of output from retrieval 
system; someon«U belWs,, identically, that retrieval systems 
can be made to operate; (W are needed, but they hav f to be . . 
accessible* so^ne $> ha, installed a s*all system, albeit crude, 
but which has now orated kr 7 years ln marginal time, on minimal 
cost, without access to qualifieo staff -.BUT with a'clear^ aim: to be 
able to find the materials wh\ch we were supposed to have.*opies of. 

■ I mentioned these last points We in practice: 

the system has to be used: therefore should not need an . 
■ ■ intermediary; it may have to depend upon non-professional staff,, 
or cheaper. staff; because, 
• ns costs should be minimal and able to compare favourably , 
with other means. of retrieving information (which may be .. 
on the habitual path of the user). 

' 'to what extent waf the demise predictabl^ There was no limit on ' 
keywording; Con/iderable -skill was required of the abstractor; a , 
iabour intensive inputting system appeared to be being .used with very. 
Ml records (though dessica.Claridge covered this by styling the 
system as "cumbersome".) „ . , 



FurZTthere was a promise to an audience which was insensitive to 
1t L.ds - or had notjyet identified', need. Clearly some people needed 
to use it. but were there sufficient? A point which we have to pursue 
.Us what is a breakeven point. How many enquiries are .acceptable to 
a critical finance officer? 



On the positive side I found some interesting points 4 which I belie\7e 
we should follow up: 

a. the self-correcting hierarchy or -Concept' map*. Can this be 
developed and work in practice? On the purest side can records 

be conflated where usage changes terms? 'Are maps of content too , 
personal or too local to allow such self : regulatfon? 

In *our own operation the draft Eudised thesaurus was analysed 

v * 

and found not to be internally "consistent. Foskett^ was unaware , 
of this. Our analysis did allow some empirical mapping £o be 
done of cbncept-maps and of changes in maps over *a period of tiijre. 

w * 

b. * The problem of strategy for inclusion - if you are generating who 

^determines the need; if you are receiving how do you stimulate^ 
appropriate input? . 

» 

c. How do -we determine hit rate or drror rate?., We used to use the 
St Mary Cray source which provided a m'assive output but most of. 
this was, irrelevant - but what should the user expect, a-precisfe 
return ot a broad return? Our own strategy is to provide a broad 
return which the professional therf sifts. What should the user 

be educated to expect? How do you get into his 'habitual pattern' 
(Don Ely)? ^ 



d. Are there ( different approaches for small scale operations as\ 
• distinct from larjge scale systems? To refer to John Cowan's paper 
ofjeveraj years ago the SCAli EFFECT'^ critic^ • The cost limits 
arejiifferent and the role of the" intermediary is anjmportant issue. 
What is the role of the librarian? Will librarians give status to 
an operation-, after all the educational building regulations specify 
;space fjsfr libraries but not for learning, staff development orcentral 
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services. Should we have large scale operations continuing or- 
make use'of small scale systems? Will the large scale operation 
be too inflexible? 



SUMMARY 



A. Is it a self confessed failure ? 

Would a search of the history/literature have prevented 
the failure 4 the cold storage ? & 



B. 



Predictable problems : no limit on keywording 

: considerable skills required of 

abstractor 

r * * 

: labour intensive t . 

: very full records 

: no needs recognised by audience 



POINTS TO 
FOLLOW UP 



/ 



: Can self-regulating trees or concept 
v*fC maps be made to work ? 

: What is error rate ? Can users be educated? . 

: Is there a SCALE .FACTOR ? 
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CHAPTER 6 



QUALIFIED CITATION INDEXING*: ITS RELEVANCE TQ EDUCATIONAL 
TECHNOLOGY 
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1. INTRODUCTION: 



tducational technology as a discipline or broad subject area, displays 
several features which form classic issues for discussion among 
information specialists. As a subject, it has no clear boundaries. 
The spread of interest crosses the path of many other specialist 
subject areas, thus making it very difficult to define from the point 
of view of information retrieval. It is a subject area which 
comprises a highly practical element at the same time as enveloping 
an element of very complex research and highly theoretical work. 
The merging of these two aspects' has already been seen through the 
literature to cause severe communication problems, and consequently 
problems of information transfer. (CLARKE & DUNN, 1977; ELY, 1979; ^ 
GILBERT, 1977; HILLS &* GILBERT, 1977; McALEESE, 1978). It is jfc' 
area within which fashions can change very quickly - what is ^ 
fashionable to study, today may be tomorrow's taboo. For example, 
television in the nineteen, seventies is now superseded by microcomputers. 
Unless one keeps up with the constantly changing emphasis, the 
terminology used in retrieving information may be totally unsuitable. 

Identification of User Population 

Although the subject spread is wide, it is probable that the users of 
educational technology infqrmation may be a select few. For exajnple, 
AETT has only 268 individual members. The user body is difficult to 
identify,, as it may be composed of people with very different subject 
backgrounds and interests. . For an information system to work 
effectively! it mdst Be in close contact. with the needs of people^who 
are trying* to obtain information from it. This involves the subject 
coverage of the base, but also to a very large extent the terminology 
used, and the inference of terms like 'information 1 for example in - 
different contexts and to several users will mean something quite 
different. A retrieval system must therefore attempt to identify the 
userpopulation and acquire as much feedback from it as possible. 
That is, thesystem should 'interrogate 1 the user to determine the 
nature of the enquiry, and permit the user to 'browse 1 in a number of 
ways. (FORD, 1977; GARVEY, 1970; BR0ADF00T, 1979; HOUNSELL, J980) . ' 
This can be a formidable problem, particularly for the larger systems 
and for any claiming international coverage. Even within a body of 
educational technologists, which a* Conference such as ETIC will 
attract, there are, one suspects* many levels of understanding of^ 
different topics, and many modes of application of apparently similah 
information. These considerations are quite independent of the mor£ 
obvious, one, perhaps, of foreign language translation'problems , which, 
apply to any subject area. One would not wish to introduce a note of 
despa^fv but educational technology does have these particular features 
of difficulty for an information system. / 



2. COMMUNICATION OF RESEARCH FINDINGS: / 



-4 ■ 
* A project supported by< the; Scottish Education Department: 1980-198*2, 
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2, COMMUNICATION OF RESEARCH FINDINGS: 



The user body has been mentioned as being small and specialised. One 
can refine this statement to mean that the body of people using the 
/literature of educational technology is small, but in a sense we. are all, 
that- is, anyone involved in teaching or communication of information 
at its simplest level, users of the products of educational technology 
and this introduces another element which is very quickly apparent to 
P anyone coming in to the subject area: the degree of communication of 
research findings* to practical situations is low. (CLARKE & DUNN, 1977; 
GILBERT, 1977), Many excellent research projects seem to find little 
support in practical application, and as corollary r . ideas springing 
from difficulties met in practiced seem to have little chance of support 
or follow-up by research for wider trial or distribution. This 
situation may well be a direct consequence of poor information retrieval 
systems, or of systems not reaching or getting through to the 
appropriate user population in the code or language of the user. There 
is very little evidence, for instance, of any effective SDI (selective 
dissemination of information) systems in educational technology , by which 
relevant selected information in some packaged form can be circulated 
to specific users or to small groups of research worker? by matchtqg 
output with pre-determined subject profiles, i,e, individualised >v 
information. 

Existing Sources of Information : 

It would be a relatively simple task to draw up^a list of sources which 
claim to covef literature on educational technology from, -say the 
British Library's collection list (KI ST - keyword index to serial titles). 
The l/test encyclopaedia on educational technology (UNWIN & McALEESE 1978) 
lists 38 'easily accessible and. useful English-language journals', of 
which four are what would be termed secondary sources, i,e, bringing 
together titles or abstracts of primary articles in meaningful subject 
groups. This is omitting some of the more general sources which will 
include educational technology, such as ERIC, with the associated Research 
in Education and Current Journals in Education, and British Educational 
Index, Another general source, which may not be so familiar in this 
field, is Social Sciences Citation Index, or even Science Citation Index,, 
both of which include much material relevant to educational technology, 
by virtue of. their cross-disfciplinary spanning. To explain further 
why this, should be so, it may be worth spending a^ittle time describing 
the principle on which citation indexing is based and its relevance to 
'educational technology, (MARTYN, 1965; WEIN§T0CK, 1971; HALL, 1970; 
MARTYN, 1975), ' ' 

Traditional Approach to Information Retrieval : 

A secondary source dealing only with literature from educational 
technology will miss many large areas, H As has been pointed oilt, in a 
subject with as wide a spread of interest, this may be the most vital 
' aspect of the retrieval, to draw from the whole spectrum of application 
in varying subject fields. Take a subject such as ' Examinations 1 
(see Fig, 1)/ A traditional hierarchical approach might be to say that 
examinations are part of the general process of Certification , which in 
turn is to do with' the purpose of Assessment ; this is one among others 
of the reasons for Assessment, others being to test curriculum, or 
competence perhaps; further, the purpose of Assessment is only part of - 
the general subject of Assessment , which is also concerned with problems 
of assessment , design of tests etc. Another user, however, might say 
that Examinations is too broad a description of his area of interest, and ; 
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might want to make subdivision of this into types of examination , 
as indicated in Fig. 1. One can see that an approach like this may 
be very helpful, particularly if the user has little knowledge erf the 
subject area, in guiding a user through the way in which the material 
in the index has been arranged. But the difficulty lies in the fact 
that this is only one way of looking at this whole subject areav . It 
is one person's or. organization's conceptual map of the subject. If 
this corresponds with the user's map, this is helpful. If, on the 
other hand, the' reason>for wanting information about 0-grade results 
was to see if, for examplef class size was an important factor, or 
whether physics should be taught to third year pupils, individually or 
in groups, for a two-hour or four-hour period, before or after lunch, 
i.e. the user's conceptual map was of a sociological nature in this 
instance, then the hierarchical type of approach may not be helpful. 

Principle of Citation Indexing : 

In an attempt to satisfy both these criteria, the approach of citation 
indexing is to assume that in many instances a user will have a 
specific problem in mind - he may know that X in Newcastle did some 
work on it, and that in fact, there was a good article defining just 
what it was, but it was several years ago and is now out of date. 
Rather than thinking of trying to define it in terms of subject 
keyowrds or patterns to match. against an existing index, the approach 
is to concentrate on the people considered to be key writers or 
workers'on the' topic. The ba'sic assumptions made are (1) that the 
key author will have written papers with lists of references, but, 
*(2)i and more importantly, that other authors or workers will have 
quoted this author in their 'lists of references. Citation indexing 
thus takes the lists of references, or cited publications, and indexes l. 
those in such a way as to link them with the authors citing or quoting' F 
them, thus working forward in time rather than back. " This can be 
seen to be a subtle* way of covering a very broad spread of linked 
subjects, since a key paper on 0-grade results, might have : been quoted 
by someone writing on achievement testing, someone AQoking at class 
•sizes and also be someone interested in, say, sociailclass and learning. 
By finding people who have quoted this paper, one calji then follow up 
similar work-in a meaningful way without the restriction of a framework 
of keywords or subject hierarchies, or language or time divisions. One 
can demonstrate this very clearly in educational technology by taking 

;, such as 'evaluation'. An educational 



a definable topic of interest 

technologist , ^knowing a little about the field, might be able to say 
immediately that there have been one or two 'key' people working in 
that area - one paper which keeps being quoted, for instance, being 
that by Parlett and Hamilton, first published in 1972 'as 'Evaluation/and 
Illumination'. By following these two key authors through a citation index 
such as the Social Sciences Citation Index*, it will be found that this paper ■ 
has 'been quoted not only by other 'authors writing on eva-luation,but by writers in 
librarian'ship, physics, computing, health, p^phology and so on. These 
papers will all have been written since the 'key '.paper, and one can choose 
approximately from the indication of journal .title, which ones will^be of 
interest. Thus one can follow a- 'key' paper forward in time. One can, 
of course, still trace backwards as well, as each citing paper will quote 
others which may also be relevant and so on. In fact, the main disadvantage 
of citation indexing is that it is almost too helpful, and that like many 
other information systems, there- is a danger of producing more information 
than a user can absorb. It does have the same disadvantage of time delay as 

* SSCI, published by the "institute for Scientific Information, Philadelphia, USA 
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traditional indexing, in that any secondary source, by definition, 
depends on first receiving and absorbing the current literature - 
it is possible, however, that the -digestion process is shorter, 
as there is less emphasis on subject division and classification. 

Qualified Citation Indexing : 

We consider that citation indexing, far the reasqns given above, is 
/ particularly suitable for application to educational technology, but 

we also feel that the selection process applied must be looked at in 
the light of the above statement on the volume of information displayed. 
The point at which bur project starts, is to'.examine ways in which a 
citation index might be refined,, and to assess, by feedback from users, 
the most meaningful ways in which this might be achieved. We are 
setting up a qualified citation. index in educational technology, 
indicating not only links between papers and those who quote. them, but- 
also the kind of link, or kind of relationship between cited and 
citing papers. It is in this sense that, the* index will be qualified - 
the nature of the relationship between jjapejfr will be indicated. To 
do this," it*is necessary to examine each reference in the context in 
which it appears and^assess/Vccording to some 'formula' the author's 
.reason for citing or 1 quoting this reference. 
Qualifiers 

We started with an arbitrary list of relationship terms or ~'qual ifiers ' , 
taken from several sources (TaBle 1). There have been several papers 
written describing systems previously attempted (LIPETZ, 1965; 
WEINSTOCK, 1971;' VQ0S, 1976-: 7 FROST, 1979: HUANG, 1968; MARGOLIS, 1967; 
OPPENHEIM, 1978), but terms were taken also from random suggestions made . 
by people in the course of their own reading. A very small pilot study 
< was undertaken by two educational technologists at this stage using 
selected articles and/personal interviews and discussion of them, and 
although this did yield more and .very Jielpful suggestions, it was 
decided that it would be extremely time-consuming and expensive to v 
do this on a larger scale at this stage. - Instead the arbitrary Vist 
« of terms drawn yi> was distributed, with introductory comment, to a 
group of lecturers and to a class of post-graduate librarianship 
students, all of wHom were assumed to be involved in reading to some 
extent! . They we/e asked to mark any term on this list which described • 
the relationship/between references and text of articles read, and in 
addition, if the! description di^not fit, p to try to describe the 
relationship between references cind text of articles read, and in 
addition, if the description did not fit, to .try to describe the 
relationship in some other way. We conducteVa similar pilot study, 
with authors of papers. We therefore ended up with a kind of league 
table of relationships, which were found to be useful, and additional? 
possibilities. This is the type of very simple consumer test we wish 
to carry out in educational technology, with as many interested users 
as possible. We have used the .'top 20' terms on articles included 
in the data base, constructed so far, and will now test reactions to,, 
these from educational technology users. Ideally, the relationships 
to be effective must be unambiguous and clear, and must be v mutually 
exclusive." They must also be sufficiently easy -for an indexer to y 
apply unequivocally, so that the total indexing time and therefore 
cost, is not increased unnecessarily. The effect of these terms or 
'qualifiers' on the output from 'the data base will be that instead of 
being able to get an answer to the question 'Who has done similar 
work to A on -this topic?', we will now be able to ask 'Has there^been' 
are view article written along the same lines as the paper by A?', or 
'Who else has used A-'s paper in a historical sense?', or 'Can I find - 
anofber 'key' paper on a topic similar to that investigated by A?'. k 



r:.- ... 7,7 
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Keyword Facility : 

A simple keyword coding of .the subject content of publications has 
also been included in our. records. This is not to be regarded as • 
a principal search strategy, but as a" secondary help, if an author 
is not known. While one is naturally looking for. Information on 
a subject, one must remember. that the subjects are in no way grouped 
or even rationalised po. any great extent - as explained previously, 
there is intentionally no requirement from the user to fit his 
subject search in to a pre-determinelr-pattern. The keywords are v 
intended to allow a user to .find out which are the 'key' authors or 
'key' papers on a topic, and taken the search b^ author from. there. 
This area is again open to discussion and modimation by interaction 
with users. 

Conclusions 

We are thus approaching what we regard as a semi -intelligent system. 
Our aim will be to produce something which will be recognisable as 
english on-line, which we can 'chat' to, or interrogate informally, 
or browse through, i.e. the user can be creative and change strategy 
on receiving further prompts, to select relevant information from 
the mounds with which we seem to be perpetually presented. • 

The aim of the Project is to produce a working computer-based index 
open to modification through discussion with users on for example 
the use and understanding of qualifiers; the definition of a 'key 
paper; the allocation of keywords and even to such physical details 
as the length of record and abbreviations used. An attempt will .also 
be made at costing the running of sO^ an index. ' 

• I 

Future: 

%*» 

As has been demonstrated by ISI, many permutations of cftation 
information are possible once a base is established, and specialist 
output, such as print-outs of rejaew articles on a specified topic; 
would be relatively simple spinoff affects. There are also 
refinements which one can carry\ out on the base itself, such as 
building in automatic counting or 'weighting' of references, so that 
assessment of various publication patterns might be made from time to 
time. - 



*************** 



A Sample Search <c SOLICIT , the system being 'developed 
at RGIT, is attached to this paper. 

V *************** , 



♦We should like to thank a number of colleagues who are 
helping us in this project: in particular John Cowan w 
Nick Rushby and Bernie Stocks. J . 
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\ : The relationship between Assessment, Examinations etc 
( subject headings taken from the SEp* Information System 
restructured by A. Lyndsay for the SED ) o 
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Bibliographical leads | 
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Review 
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Correct ion 

Criticism 

Disputing 
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Historical 
Background reading 
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Narrative 
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Definition 
Illustration, 




Corroboration ^ 
Similar/research 
Similar work 
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Anticipated value 


Statistics ^ 
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Data , 




New research • , 
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Practical application 



Table 1: Relational Qualifiers - (i.e. descriptions of the 
type of relationship between a reference quoted 
by an author and the context of the quotation). 
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On the scope of Edtech |* * 

Perhaps naively I'vi thought educational technology j>r edte'ch was concerned 
with the knowledge, {use and effectiveness of equipment applied to 
educational .techniques - not really concerned with theory and experimentation 
in eg examination arid assessment.- while one cannot divorce the science pf 
the subject from its\ technology I should not expect edtech to be concerned 
•wifh the "science" extept as related to the use -or effects of technical 
methods or devices. X I had a look at some edtech books on my library's 
shelves and see that there are divergent schools of thought on this matter 
of the scooe of edtech within the education '-field. 

V 

In the paper I ran see some relevance for "Education and illumination" eg 
Parlett & Hamilton's work, when considering the effectiveness of some 
project using edtech techniques - 'graphics, 'slides*,' tapes ... video ... 
as this -wffs. important and indeed much argued about? in a major project I 
vfas, involved' i ri in the library/information field viz The Travelling 
Workshop Experiment - and colleagues on this made good contribution to 
this subject. I poser' a question later to get your views* - t 

On the quoted use of research in edtech by its practitioners I had a-look ( , 
at another of Dr. McAleese's papers in the journal SPE (Scientia Paedagogi^a 
Experimentalis). While we find that research in the library/information ' 
field is also ignored by most practitioners - goodnir^significant research 
ideas or findings do get attention from the conscientious - who in any 
field, let's face it, tend to be a, minority^ Dr. McAleese has some 
interesting quotes in his paper (SPE). 



"60% of those who publish research papers do so on 
a one-time basis .\ ." 



/ 



and for reviewers of research in a particular topic ^ 
"'an overly larfce proportion of the data ... must be / 

• ■ . / 



considered somewhat -trivial 



again "At the present most of the research seems 
confused .:." , - 

1 again and I feet 'a kinship with the information field - 
/' "inductive re^nirfg, hypothesis testing, scientific / 
/ methodology may not be appropriate to the social 
* sciences .w . . .assumption? about variables n6kes mocjel 
building suspect .... / 
» * 

Moving on to Citation indexing - . « . .• 

• To the edtech practitioner or 'researcher' - ^ 

as fn any subject field an important method of getting into -the literature 7 
is following up a known paper- whether found by serendipity or from, 
a colleague - this tends to b& preferred to using subjeqt indexes or ^ 
abstracts - or libraries/information people.. John Martyn's 1965 Aslib' 
paper touches on this 7 if you have time or inclination it is as good 
an entry to what citations are about as you'll find.- « ( » \ 
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The prime examples of citation indexing are the massively impressive * 
publications of the Institute for Scientific Information - the Garfield 
empire-which combine all three approaches to retrieval - the contents 
lists of pertinent journals and books, the keywords of titles as subjects, 
and citation following ie the Kg/ere^hces given in papers by their. authors. 



ISI's files have over 7 pillion references. Handling 



level of .citation indexing indeed 
is impractical , 



must use computers 



such - and any 
- manual handling 



ISI 's 'citation indexing is unqualified. Ingenious means are^used to. 
reduce the labour/cost intensive operations of analysing the contents of 
thousands of journals including many in tf^e^edtech field - however you 
choose to define that. One is Project Keysave which checks all new 
citations against a huge online file - and over 60% of new input is found 
to have, been used in the file before (sqe p. 42 vol. 3 Essays). 

Still on methodology - citation indexing has to be continuous - if say 
there are 500 references on a particular topic over a,5-year period maybe 
30% will be reared to, i.e. cfted *n one subsequent'year. 

You can by "cycling" catch a large proportion of the 500 by examining 
the references in this 30%. 

So an edteCh citation index as an ongoing system needs to be very sure of 
subjett scope required and the range of /sources that suit its intended 
users. L haven't looked in detail at ISI's coverage of edtech fcjr-UK? 
users - naturally there 's^some. US bias \nd it doesn't stretch to minor . 
journals and newsletters or trade/technical- and availability of; its 
pr|nted form is presently. very limited 

A feature of cjtation following which this project seeks to alleviate is 
"Noise" .' If you check out the titles of "cited" papers 'retrieved by 
a citation index maybe only one-third will be subject relevant - and that > 
applies whether the subject specialist or a library/information intermediary 
looks at the titles. * ' 

I was astfed to pose some 'questions to v entice you to react to this exciting 
•subject - I have seven - 



,1. . On Scope 



4 



v. 

1A • 



For/you are 



'For you 



2. Retrieval 



Examinations 
Certi fi cation 
Assessment 

Is analysis of 
'0 1 grade r^ults 
2 or 4 hour periods 
1/jnch hpurs* . 



-edtech?' 



ito 



, -edtech? 



up 



Do you prefer following 
References/citations . 
Using subject indexes/abstracts 



•3. Usefulness 



Fallowing 'up selected citations 

Are you frustrate^ by many which are just "paying homage 
or "ni^torical" . v ' 



Frustration , « . ' - 

■ When y.pu see a reference .like SCIENCE 155., 1 
. Do you react' "What- year?"* , 



Getting 



How much frustration from trying to 
get a copy? 



You 



When/ you write something - do yo'u Q make *he # 
titlB-Jiieanirigful and clear? V 
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A Sample Search on.SOUCIT, .the Qualified Citation Index 
being developed at 'RGIT . 



•f^'O Uses response 

' 1 \ 



• O run soucit 

Have you used this program before? Y or . 

On ' 



This is* a Qualified Citation. Index, that is entries' 
have been made from bibliographies of written papers 
indicating links- between publications quoted in the 
bibliography, or list of references (cited publications) 
and the, context in wh v ich they have been quoted, 
(citing' publications) . ^ • » 
The Index may be searched eitBer by Author ( A ) , . 

^ t ' or by subject keyword ("B") . 

If searching by author there are a number of alternate 
approaches which will be explained during the search. 

Which approach do you wish to take first, "A" or "B"? 
Please type "A"<PETUFN> or* "B"<EETUHN>'. ( % m 

> a * 

Please give the Author's name you wish to 'search on: 

(for. example "HA^LTCN"<PJETUFN> ) 
The Program will accept the correct Name if you 'already 

know it.' -.' * ' , 

(for example "HAMILTON D" ) 

Authors Name? ♦ * • 



in this form. Similar names are... 



ham 

Name not found 
HAMILTON D* ' 
HAMILTON F - 

HAMILTON H . , ' 

Authors Name? . , 

hamilton d . - „„ M , tM n 

Do you 'want biographical information on this aushor? Y, or N 



O y 



DEPARTMENT OF EDUC. ,UNIV.OF GLASGOW 
In the Index aire listed. publications- by the Author as . 
principal or joint' author , and items which cite One or more 
publications by<the Author. Yo'u may choose to see all publications,- 
in which, case you will be' .given a straight list. If you wish •* 
S see the lists of references attached ^ each of the publications 
you should choose "G" (CITING) . _ 
If you wish to do a Citation search C?B") 

•you 'will be given a list of publications by this,author which 
have been CITED, by others, with. details *of the citing authors 
and relationships between the papers in each case; 
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* Input "A" to see all publications' by tins author (ALL) 
Input "B" to see items by this author which, have been 
• • cited, by others (CITED) , , t , 

Input "C" to see items by^this author with their • 
lists pf references. (CITING) / < v ' ..*:„*. 

^"7 There are 10 publications by . ^ 

Do you wish to see 'them ail?,' . 
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You may limit the list in 'one of the following ways: 

1. By YEAR 'e.g. only publications in the last 3 years 

2. By\$YPE OF PUBLICATION e.g. only review articles ' 
J. By SUBJECT KEYWORD 

4. By the NUMBER OF TIMES a paper has been CITED e.g. the 3 
/ papers most frequently referred to by others 

5. By selecting KEY papers - papers which have been given 

a iDarticular weighting . - 
V 

Indicate your choice of limitation by typing the appropriate number. 

- O 1 .• 

Type the year or range of years- (separated by a space) from which you • 

v wish to see publications. J 

* CD 1976 1977 

HAMILTON D 

1976 ' • . * . r 

CURRICULUM EVALUATION ■ 

HAMILTON D ) 
i *• 1976 , , 
IN SEARCH OF STRUCTURE: ESSAYS FROM AN OPEN PLAN 



J- 



HAMILTON D ' • 

1976 

SOME CONTRASTING ASSUMPTIONS ABOUT SURVEY ANAL 



HAMILTON D 
1977 

BEYOND THE NUMBERS GAME :.. EDITED BY D HAMILTON. . 



' . HAMILTON P . , 
. '1977'. ■ . ' • 

EVALUATION AS ILLUMINATION 

>. ' - , ,• 

HAMILTON D • ' 

1977 ' 

*• . ILLUMTNATIONS AND RUMINATIONS 1 # . .. 

' ' ' . • ' ' ' i 

'. * 

HAMILTON D 
* • « _. 1977 

' IN SEARCH OF STRUCTURE » 

Do you wish for more bibliographical . information on any one publication? 
*Y' or 'N' , . .* ' ' 

•Please -type "Year" and "First five characters of title". 
r (for example: "1976IN^E") 

O 1977bevon * ■ v > „, 

, ^ " BEYOND THE NUMBERS GAMET: A READER IN EDUCATIONAL _ 

EVALUATION. EDITED BY DAVID HAMILTON; .DAVID. JENKINS 
0 « - CHRISTINE KING? BARRY MAGDONALD; MALCOLM- PARLETT * 

ERIC 

""«a No further information 



Do you wish for a citation .search on any one publication? '¥i or 'N' 

s o y v '*• 1 

Please type "Year" and "first five t characters of title." 
• O 1977in se % * 

Cited by <> 
«_ *HAMILTON D 

. ' 1977 

ILLUMINATIONS AND RUMINATIONS 
• IN- RESEARCH INTELLIGENCE 3(1)22-23 
t Article 

. Background reading * 
Methodology 
• • No further information. 



Authors Name? . . " 

O hamilton d 

Do you want biographical information on this author? Y or TJ / 

Input "A" to see all publications fty this author (ALL) 

"input "B" to see items by this author which have been ^ 

cited by others (CITED) 



^ TherS are 8 publications**^ -his author which have been cited hy others. 
Do you wish to -see then all? ' * J . 

Option unavailable at^preserif: 

Authors Name? v > • • * 

hamilton d * • ■ ' ' * * 

Do you want biographical information on this author? Y or ^ * 

Input "A" "to see all oobiications by this author (ALL) 

Input "B" to see items by this author which have, been > 

. cited by others {CITElft '. m 

• Input to see. items by tfcis author with their 
lists of references (CITING) ' 

O b " j . > * ' c 

Thefe,are 8 publications by this author Which have been cited by otfters. 

'Do you wish ,to see them all? • 

O y 



1972 * ' / 

EVALUATION AS ILUJMINATICN 

CENTRE PES. EDX.SQ.roiNBUBGH.OCCAS. PAPER 9 



\ 



Cited by . " N' . \ 

• • • *PARLETT M 

1977 - . • . 

' EVALUATION AS ILLUMINATION 

.- . • ^ ( IN-BEYCND TOE NUMBERS GAME. EDITED BY D HAMILTON 

* ** > . -Article 

— p^. * Conference Paper 
■Cited by x ' « 

♦HAMILTON 0 
„ 1977 ' «■ 

0 ILLUMINATIONS 'AND RUMINATIONS • 

CDJ£- * IN^RESEARCH INTELLIGENCE 3(1)22-23 

Hm^an ' • Article- " .on 

« Paying hcrnage UvJ 
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Cited .by , 

*FJALLBRANT N . . 
1977 f ; 

EVALUATION IN A USER EDUCATION PROGRAM 
IN-JOURNAL OF LIBRARIANSHIP 9(2)83-95 

Article 

Definition 

Cited by 

* HEWITT C 
1978 

■ EVALUATION ^ * 

IN-ENCYCL.EDUC. MEDIA COMMUN^.UNWJN. PP. 338-346* 
v j- Article j 

Paying -homage ^ 
Biblioqraphical leadp 
Clarification * 

; / 

' 1974 * » . 

CLASSROOM RESEARCH: A CAUTIONARY TALE. BY D HAM. 
IN- RESEARCH IN EDUCATION 11 1-15 



Cited by 

* MCALEESE R 
1978 * 

UNDERSTANDING CLASSROOM LIFE. BY R MCALEESE- & D. • ■ 
SLOUGH, NFER PUBLISHING CO. 
Article m 

Disputing , 
Similar research 

■ &>$ c . . . i , 4 

Abthors Namef? ) 
O hamilton d • • 
Do you want biographical information on this author? Y or N 

Input "A 11 to see all publipations by this author (ALL) v 

• Input "B" to se^ items by this author which have been 4 
cited by others (CITED) " 

Input "C" to se<? items by this author with their 

list^'of references (CITING) - * 

O c / • * a 

' There are*3 publications by this author with lists of references. 

• Do you wish to see then all? * 

O y 1 



1977 * . 

EVALUATION AS ILLUMINATION • 
' IN-BEYOND THE NUMBERS' GAME. EDITED BY D. HAMILTON 

M 

• i 

♦ C f 

, Citing ■ * • . 

*PARLETT M \ ' 

,1972 / * * s 

EVALUATION AS ILLUMINATION ' , © 
CENTRE RES. EDUC. SCI . EDINBURGH. 0CCA6. PAPER 9 
Article 

Conference Paper 



UNDEPSTMJblNG CLASSt»OM LIFE. BY R MCALEESE & D- H 
SLOUGH, *NFER PUBLISHING CO. 



Citing 



Citing 



Citing 



Citing 



♦MCALEESE R 

1978 ; 

NATURE AND DISSEMINATION OF EDUCATIONAL TECHNOL. . 
SAN FRANCISCO , NAT . SOC . PERFORMANCE & INSTRUCTION 
Article 

Experimental detail - ^ 

Development of ideas 

♦HAMILTON D ^ 
1974 

CLASSROOM RESEARCH: A CAUTIONARY TALE. BY D HAM... 
. " IN- RESEARCH IN EDUCATION 11 1-15 f 
Article 

Disputing ' ~ 

Similar research 

*HOBBS S 
1976 , 

* CIP TEACHING ANALYSIS MANUAL 
GLASGOW, JORDANHIIjL COLLEGE 
Article 
Theory 

Similar research 



*SK INNER B F ' 
1969 

CONTINGENCIES OF REINFORCEMENT 
ENGLEWOOD CLIFFS , PRENTICE-HALL 
t Article 

' Background reading 
Definition 



/ 



' 1977 

ILLUMINATIONS AND RUMINATIONS 

IN- RESEARCH INTELLIGENCE 3(1)22-23 



Citing.' 



*CRONBACH L J 
1975 



^ BEYOND THE TWO DISCIPLINES OF "SCIENTIFIC PSYCHOL 
IN-AMERICAN PSYCHOLOGIST 30(2)110-127. 

' Article' . 

Conference Paper * • _ / » 



f ■ 



3 



91 
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9.55 Task on identifying individual needs in information retrieval 
lp.05 D. iSly: The information-seeking behaviour of users 
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10.40* Reply to D. £Ly. N. Rushby 
10.55 DISCUSSION 

11.10 J. Hustwit: Information services in practice 
11.25 Reply. M. Head 

11.35 DISCUSSION 

-II.50 irking Information Systems: ; 

1. W. Davies: The use of Query in multi-»facet 

indexing of information materials ♦ 

2. J. Claridge: The Exeter Abstract Reference 

System- on higher education 

By D. Bligh * : 

12.20 Reply. M. Roebuck 

12.30 DISCUSSION « : 

, • 
12.45, . * HNCH" 5 *"**"**"**"**"^ 

2.00 £.* Duncan; Qualified citation indexing - its relevance to 

* °* 1 educa*tiojti£[l technology. By R. r McAleese; D. Anderson; 

iS. Duncan * A 

2.20 Reply/ L. Corbptt 0 & & " * 

2.35 DISCUSJION° ° > 

2.50 Formation of Working Groups * <> 

The group task will b3 to identify ways in which information n€jeds 
may best be 'fulfilled in one of three specific &eas 
GROUP DISCUSSION % . ■ . * 

o 20 " * ^hhhh^^h^O FFEE^^^^ 

* • % 

3.U0 GROUP DISCUSSIONS (contd), ■ ' 

» ♦ 

4.. 15 ^Report back from groups. , ^ . * 

Synthesis • 
5.00 Finish " 92 -■*' % " ' 
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